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Abstract 

Pilots  of  vertical  landing  vehicles  face  numerous  control  challenges  which  often  involve 
the  loss  of  outside  visual  perceptual  cues  or  the  control  of  flight  parameters  within 
tight  constraints.  These  challenges  are  often  associated  with  a  high  mental  workload, 
therefore,  a  precision  landing  aid  that  addresses  and  helps  to  mitigate  these  challenges, 
and  reduce  mental  workload  is  needed.  To  address  this  need,  a  cognitive  task  analysis 
identified  specific  situation  awareness  requirements  for  the  design  of  a  vertical  landing 
aid  in  order  to  reduce  the  mental  steps  required  during  a  vertical  landing.  From  these 
requirements,  a  new  vertical  landing  decision  aid,  known  as  the  Vertical  Altitude  and 
Velocity  Indicator  (VAVI)  was  designed,  which  displays  altitude  and  vertical  speed 
information  in  an  integrated  form  including  the  display  of  flight  parameter  safety 
constraints.  The  display  instrument  takes  advantage  of  direct-perception  interaction 
by  leveraging  ecological  perception  and  emergent  features  to  provide  quick  perception 
and  comprehension  of  critical  flight  parameters  in  an  integrated  fashion. 

To  test  the  effectiveness  of  the  VAVI  for  vertical  landing  and  hover  performance, 
an  experiment  was  conducted  in  which  participants  flew  a  simulated  Harrier  vertical 
landing  flight  profile  using  Microsoft  Flight  Simulator  (MSFS)  2004.  Participants 
were  recruited  for  their  helicopter  pilot  experience  or  PC  flight  simulator  experience. 
Two  heads-up  displays  were  implemented:  one  which  included  the  VAVI,  and  another 
which  displayed  altitude  and  vertical  speed  information  consistent  with  operational 
V/STOL  aircraft  head-up  displays.  A  2x2  ANOVA  design  was  utilized  in  which 
the  heads-up  display  was  a  between-subjects  factor  and  flight  task,  which  included 
hovering  and  landing,  was  a  within-subjects  factor.  Participants  participated  in  two 
test  scenarios  which  involved  hovering  at  a  specified  altitudes  and  descending  using 
either  a  static  or  dynamic  vertical  speed  heuristic. 

The  VAVI  showed  statistically  significantly  better  vertical  speed  control  perfor¬ 
mance  over  the  conventional  display  of  altitude  and  vertical  speed.  Similarly,  though 
not  statistically  significant,  other  dependent  variables  used  to  measure  landing  per¬ 
formance  as  well  as  precision  hovering  consistently  resulted  in  better  performance 
with  the  VAVI.  A  subjective  workload  survey  indicated  that  the  VAVI  caused  less 
workload  across  all  experimental  tasks,  indicating  that  the  VAVI  does  help  to  remove 
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some  of  the  demanding  cognitive  processes  currently  associated  with  vertical  landing 
and  hover  operations.  Future  design  and  implementation  issues  are  discussed. 

Thesis  Supervisor:  Mary  L.  Cummings 

Title:  Assistant  Professor  of  Aeronautics  and  Astronautics 

Thesis  Supervisor:  Laura  M.  Forest 

Title:  Human-Machine  Collaboration  Engineer 

The  Charles  Stark  Draper  Laboratory,  Inc. 
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Chapter  1 


Introduction 


The  need  for  vertical  precision  landing  capability  has  long  been  recognized  for  mili¬ 
tary,  space,  and  commercial  applications.  While  the  first  vertical  takeoff  and  landing 
aircraft,  such  as  balloons  and  airships  were  difficult  to  maneuver,  current  technology 
has  revolutionized  air  and  spacecraft  vertical  flight  and  the  future  of  this  capability  is 
promising.  The  advantages  of  vertical  flight  do  not  come  without  unique  challenges 
however.  Specifically,  vertical  operations  such  as  hovering  and  precision  landing  are 
difficult  tasks  that  can  be  confounded  by  various  aerodynamic  effects  or  a  loss  of 
visual  cues.  This  is  a  challenge  because  for  vertical  precision  landings  in  particular, 
pilots  rely  almost  completely  on  perceptual  cues  external  to  the  cockpit.  Precision 
landing  flight  displays,  however,  have  not  caught  up  to  the  promising  advances  in 
vertical  flight  technology. 

Aircraft  with  vertical  takeoff  and  landing  capability  fall  into  one  of  two  categories: 
V/STOL  (vertical/short-takeoff  and  landing)  and  helicopters.  V/STOL  aircraft  in¬ 
clude  convertiplanes  such  as  harriers  (a  vectored-thrust  aircraft)  and  Ospreys  (a  tilt 
rotor  aircraft)  which  can  fly  horizontally  with  the  same  effectiveness  as  conventional 
aircraft,  but  have  the  capability  to  takeoff  and  land  vertically.  Helicopters  are  rotor- 
crafts  that  derive  their  lift  from  rotating  blades  regardless  of  the  phase  of  flight.  The 
Harrier  and  V-22  Osprey  are  the  only  operational  V/STOL  aircraft  used  today,  with 
the  Joint  Strike  Fighter  (JSF)  soon  to  become  the  next  generation  V/STOL  aircraft. 

In  the  space  domain,  the  Apollo  Lunar  Module  (LM)  was  designed  for  vertical 
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landings  and  takeoffs  on  the  Moon.  During  the  Apollo  era  (1960s-1972)  there  were 
no  high  fidelity  maps  of  the  Moon  and  thus  a  vertical  landing  was  exploited  to  ensure 
there  were  no  hazards  in  the  landing  region.  Current  studies  indicate  that  future 
missions  to  the  Moon,  and  later  to  Mars,  will  most  likely  have  a  similar  vertical 
landing  trajectory  [1], 

The  vertical  descent  to  landing  represents  one  of  the  most  difficult  aircraft  ma¬ 
neuvers  that  hover  aircraft  pilots  face.  It  was  also  one  of  the  most  challenging  and 
unique  aspects  of  landing  on  the  Moon.  Lunar  and  earth  vertical  landings  exhibit 
several  similar  perceptual  challenges  with  few  tools  to  assist  the  pilot  with  such  tasks 
as  hovering  and  maintaining  a  safe  descent  rate,  while  sustaining  an  accurate  aware¬ 
ness  of  the  present  state  of  the  vehicle.  One  of  the  primary  technologies  available  to 
air  and  spacecraft  pilots  for  making  a  precision  landing,  especially  in  low  visibility 
situations,  are  integrated  flight  instrument  displays.  Integrated  flight  displays  pro¬ 
vide  critical  vehicle  attitude  and  rate  information  with  a  higher  degree  of  precision 
than  the  human  can  discern  through  visual  and  vestibular  cues.  This  thesis  examines 
the  human  factors  challenges  of  precision  vertical  landing,  for  both  air  and  space  do¬ 
mains.  While  takeoff  is  also  an  important  flight  phase,  this  thesis  focuses  specifically 
on  the  challenges  of  precision  vertical  landing.  A  new  precision  landing  aid  that  lever¬ 
ages  ecological  perception  is  introduced  and  its  effectiveness  tested  and  discussed  for 
application  in  both  domains. 


1.1  Motivation 

1.1.1  Aircraft  Vertical  Takeoff  and  Landing  Capability  Ad¬ 
vantages 

V/STOL  aircraft  and  rotorcraft  have  many  advantages  in  today’s  military  as  well 
as  first  response  and  commercial  applications.  The  ability  to  operate  from  rapidly 
constructed  expeditionary  airfields,,  forward  sites  such  as  roads,  various  amphibious 
ships,  and  damaged  conventional  airfields,  enables  dedicated  close  air  support  [2]. 
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In  addition,  not  requiring  a  runway  enables  V/STOL  aircraft  to  reach  very  exact 
areas  for  the  purposes  of  search  and  rescue  operations  and  airlift  or  airdrop.  With 
urban  warfare  becoming  increasingly  more  common,  the  ability  to  reach  tight  spaces 
such  as  the  tops  of  buildings  or  roads  between  large  structures  is  even  greater.  In 
these  situations,  the  ability  to  vertically  land  precisely  and  safely  becomes  a  critical 
factor  in  the  success  of  such  missions.  In  non-military  applications,  helicopters  and 
V/STOL  aircraft  are  advantageous  for  their  efficient  use  of  airspace,  reduction  in  the 
noise  footprint,  and  the  ability  to  land  in  weather  conditions  prohibitive  for  fixed- 
wing  aircraft  [3].  The  low  landing  speeds  of  V/STOL  aircraft  allow  more  time  for 
pilot  decision  making  and  maneuvering,  which  may  be  favorable  during  instrument 
conditions  [4,  3]. 

1.1.2  Lunar  Lander  Requirements 

The  need  for  precision  landing  on  the  Moon  or  Mars  is  driven  by  the  uncertainty 
in  the  terrain,  as  well  as  the  requirement  to  be  able  to  land  near  another  vehicle  or 
infrastructure,  or  be  able  to  reach  specific  locations  of  interest  from  the  landing  site.  In 
support  of  the  United  States’  vision  to  return  humans  to  the  Moon  and  Mars  by  2012 
for  novel  exploration,  the  next  generation  Lunar  lander  must  be  capable  of  achieving 
pinpoint,  anytime,  anywhere  safe  landing  on  the  Lunar  surface  with  high  precision 
(10-100m)  [1].  Current  work  towards  this  goal  indicates  that  a  vertical  landing,  similar 
to  that  of  Apollo,  will  most  likely  be  used  for  landing  humans  and  equipment  on  the 
surface.  While  the  Apollo  trajectory  included  a  vertical  descent  from  roughly  50  feet, 
which  allowed  the  crew  to  preview  the  landing  site  upon  approach  and  then  descend 
on  to  it,  the  proposed  trajectory  for  returning  to  the  Moon  may  include  a  more 
distinct  vertical  descent.  This  is  to  allow  sensors  to  conduct  hazard  detection  and 
avoidance  during  the  vertical  descent,  and  create  a  synthetic  view  of  the  landing  site 
for  the  crew  [1].  Figure  1-1,  not  drawn  to  scale,  illustrates  the  comparison  between 
the  Apollo  landing  trajectory  and  a  proposed  return  trajectory.  The  requirement  for 
a  vertical  landing  motivates  the  need  for  improved  landing  aids  over  those  that  did 
exist  for  the  Apollo  Lunar  landings. 
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Powered  Descent 


(a)  Apollo  Landing  Trajectory  (b)  Future  Moon  Trajectory  (adapted  from  [1]) 


Figure  1-1:  Moon  Landing  Trajectories 

1.1.3  Vertical  Flight  Challenges 

Numerous  challenges  accompany  the  ability  to  conduct  safe  vertical  operations  in  air 
and  spacecraft.  An  understanding  of  these  challenges  will  provide  an  appreciation 
for  the  periods  of  flight  during  which  precision  and  safety  is  most  applicable,  when 
instruments  and  displays  are  most  utilized,  and  why  a  perceptual  precision  landing 
aid  is  needed  by  pilots  of  vertical  air  and  spacecraft. 

Ground  Effect 

One  challenge  that  pilots  face  is  ground  effect,  which  is  created  when  the  ground 
interrupts  the  flow  of  air  around  part  of  a  vehicle  [5].  For  helicopters,  ground  effect 
is  generally  a  positive  effect  that  improves  performance  when  the  aircraft  is  near 
the  ground  (within  about  1/2  of  the  rotor  diameter)  because  of  the  reduced  velocity 
of  downward  airflow  and  the  reduction  in  rotor  tip  vortex  [6].  Since  the  airflow  is 
interrupted  by  the  ground,  its  velocity  is  reduced  when  flowing  back  down  through  the 
rotor  disk.  This  allows  the  lift  to  increase  and  thus  improves  performance.  Essentially 
it  creates  extra  lift  which  means  the  helicopter  requires  less  power  to  hover  in-ground- 
effect  (IGE). 

When  a  helicopter  is  farther  from  the  ground,  the  circulation  of  air  through  the 
rotor  disk  and  back  around  can  cause  turbulence  and  instability  if  the  vortex  swirls 
are  too  large.  Therefore,  out-of-ground-effect  (OGE)  hovers  require  more  power  and 
are  often  more  unstable  making  it  a  difficult,  though  sometimes  neccessary,  task. 
Because  of  this  instability,  helicopter  pilots  report  that  they  are  more  concerned  with 
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their  instruments  and  vehicle  state  during  this  phase  of  flight.  Figure  1-2  illustrates 
the  difference  between  in  and  out-of  ground  effect. 


OUT-OF  GROUND-EFFECT  HOVER. 

(a)  In-Ground  Effect  (b)  Out-of-Ground  Effect 

Figure  1-2:  Ground  Effect  (from  [6]) 


On  the  other  hand,  IGE  can  be  unfavorable  for  some  V/STOL  aircraft,  depending 
on  the  configuration,  because  it  sometimes  causes  a  suction  effect  that  pulls  the 
aircraft  to  the  ground  [3].  Jet-lift  V/STOL  aircraft,  such  as  the  harrier,  sometimes 
experience  this  effect.  In  a  study  conducted  using  the  NASA  Ames  V/STOL  Systems 
Research  Aircraft  (VSRA),  which  is  similar  to  a  Harrier,  pilots  were  asked  fly  a 
decelerating  approach  to  a  hover  followed  by  a  vertical  landing.  Afterwards  they 
were  asked  to  rate  the  handling  qualities  for  a  vertical  landing.  While  the  ratings 
were  adequate,  the  pilots  noted  that  they  considered  the  principal  deficiency  of  ease 
of  landing  to  be  “the  considerable  attention  and  compensation  required  to  control  sink 
rate  during  the  descent  in  the  presence  of  ground  effect”  [7,  p.48]  -  a  stark  contract 
from  the  experiences  of  helicopter  pilots  IGE. 

Vortex  Ring  States 

Another  challenge  is  vortex  ring  state  (VRS),  which  is  applicable  to  rotorcraft  such 
as  helicopters  and  the  V-22  Osprey.  Entering  a  vortex  ring  state  is  often  referred 
to  as  “settling  with  power”  and  occurs  when  the  rotorcraft  descends  into  its  own 
vortices,  or  downwash,  disrupting  the  flow  that  creates  lift.  The  VRS  can  be  initiated 
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by  descending  too  quickly  (generally  >  300  fpm)  while  applying  power  and  traveling 
at  low  airspeeds  (typically  considered  <  30  knots)  [8].  Entering  a  vortex  ring  state 
causes  a  loss  of  control  and  of  stability,  which  is  irrecoverable  in  extreme  cases.  For 
this  reason,  rotorcraft  pilots  are  especially  concerned  with  their  descent  rate  during  a 
powered  vertical  landing  and  depend  on  their  instruments  in  addition  to  their  visual 
cues  to  determine  this. 

Numerous  helicopter  and  Osprey  mishaps  have  been  attributed  to  VRS.  In  April 
2000,  a  simulated  non-combat  evacuation  mission  including  four  MV-22  Ospreys  from 
the  Marine  Aviation  Weapons  and  Tactics  Squadron  One  (MAWTS-1)  ended  in  a 
mishap  of  one  of  the  Ospreys  and  the  loss  of  all  15  Marines  and  4  crew  members 
aboard  due  to  a  VRS  situation  as  a  result  of  a  descent  rate  exceeding  the  flight 
envelope  [8].  Figure  1-3  illustrates  the  airflow  through  rotor  blades  for  normal  hover 
operations  versus  those  experiencing  a  VRS.  As  illustrated,  in  the  VRS  the  induced 
flow  is  upward  through  the  inner  portions  of  the  blades  thus  reducing  lift. 


(a)  Hovering  Flight  (b)  Vortex  Ring  State 


Figure  1-3:  Vortex  Ring  States  (from  [9]) 


In  some  cases  it  may  be  impossible  to  recover  from  a  VRS,  especially  if  the  aircraft 
does  not  have  the  necessary  altitude.  In  a  helicopter,  a  pilot  can  try  to  recover  from 
a  VRS  by  lowering  the  power,  pitching  the  nose  down  and  attempting  to  gain  some 
forward  velocity.  Similarly,  the  Osprey  can  recover  by  rotating  the  nacelles  (the 
double  rotor  blades)  to  a  more  horizontal  position  in  order  to  gain  forward  velocity. 
In  both  cases,  once  airspeed  has  increased,  an  increase  in  power  is  necessary  to  cease 
the  fast  descent  rate  [8]. 
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Meteorological  Flying  Conditions 


While  flying  in  adverse  weather  conditions  is  never  desirable,  it  is  sometimes  necces- 
sary.  In  the  aviation  domain,  weather  drives  the  procedures,  which  are  defined  as 
meteorological  flying  conditions.  The  procedures  are  primarily  a  function  of  visibility 
or  how  far  ahead  the  pilot  can  see.  Visual  meteorological  conditions  (VMC)  enable 
visual  flight  rules  (VFR)  to  be  used  if  there  is  a  clear  visual  of  the  natural  horizon 
and  the  ground  that  can  be  used  as  a  reference  [10].  Pilots  still  use  their  instruments 
as  references  during  VFR  flight,  but  the  outside  world  provides  the  primary  attitude 
and  rate  cues. 

Instrument  meteorological  conditions  (IMC)  are  those  conditions  in  which  instru¬ 
ment  flight  rules  (IFR)  are  required.  IFR  is  flight  with  reference  to  the  instruments 
only  and  is  neccessary  when  weather  causes  low  ceilings  and  visibility  [10].  A  ceiling 
is  an  overcast  layer  that  precludes  a  pilot  from  traveling  through  it  under  VFR.  Fog, 
haze  and  smoke,  dust  and  sand,  and  precipitation  can  cause  IMC  [10].  Helicopter 
and  V/STOL  aircraft  pilots  rely  strongly  on  their  outside  visual  cues  to  determine  a 
change  in  position  and  compensate  for  that  change  through  control  inputs  [11],  so  a 
loss  in  visual  cues  requires  that  instruments  be  designed  in  such  a  way  as  to  provide 
similar  information  in  an  intuitive  manner.  As  of  now,  however,  only  high-workload 
tools  are  in  place  to  aid  pilots  with  hovering  without  those  visual  cues.  Flying  under 
instrument  conditions  is  necessary  to  be  able  to  exploit  the  full  operational  advan¬ 
tages  of  helicopters  and  V/STOL  aircraft,  such  as  operating  from  remote  sites  and 
landing  in  confined  areas  [4],  Once  instrument  rated,  Harrier  pilots  can  fly  in  weather 
down  to  200  feet  ceilings  and  one-half  mile  of  visibility  using  IFR. 

Nighttime  flying  can  be  done  using  VFR  or  IFR,  however  it  presents  several  unique 
challenges.  When  using  VFR,  visual  illusions  affecting  the  determination  of  height, 
distance,  and  identification  of  stars  or  lights  in  the  distance  can  severely  influence 
the  pilot’s  situation  awareness.  According  to  a  mishap  survey  for  the  U.S.  Army 
published  in  1999,  31%  of  all  U.S.  Army  rotary-wing  mishaps  between  1987-1995  were 
due  to  spatial  disorientation  [12].  Most  of  these  mishaps  were  caused  by  unrecognized 
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spatial  disorientation  and  occurred  at  night,  when  visual  cues  are  not  as  prominent. 
Unfortunately  visual  and  vestibular  cues  do  not  always  provide  accurate  information, 
making  flight  instruments  and  displays  critical  tools  during  nighttime  flying. 


Visual  Disruptions 


Closely  related  to  meteorological  flying  conditions  is  the  issue  of  brownout  or  whiteout 
conditions.  Brownouts  (also  known  as  dustouts)  and  whiteouts  are  the  result  of  the  air 
movement  caused  by  the  rotors  or  jet  exhaust  close  to  the  ground  which  can  create  a 
cloud  of  sand  or  snow  which  significantly  reduces  visibility  during  the  final  portions  of 
a  vertical  descent  [8].  An  example  of  brownout  conditions  is  illustrated  in  Figure  1-4. 
In  the  United  States’  current  major  engagement,  Operation  Iraqi  Freedom,  multiple 
aircraft  and  lives  have  been  lost  due  to  failed  landings  in  dustouts  [13,  14].  Dustout 
conditions  create  a  similar  problem  as  night  or  IMC  flying,  which  can  be  alleviated 
to  some  degree  through  the  use  of  instruments  and  flight  displays. 


Figure  1-4:  Brownout  Conditions  (from  [15]) 


Dustouts  also  posed  a  problem  during  the  Apollo  Lunar  landings.  While  all  of 
the  Apollo  landings  experienced  some  dust,  some  landings  were  more  obscured  than 
others.  The  Apollo  15  landing  was  one  of  the  worst  dustout  conditions  that  the 
astronauts  encountered.  Mission  commander  Dave  Scott  described  the  dust  during 
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the  landing  when  he  stated  that  “At  about  50  to  60  feet,  the  total  view  outside  was 
obscured  by  dust.  It  was  completely  IFR.  I  came  into  the  cockpit  and  flew  with  the 
instruments  from  there  on  down”  [16,  p.l].  The  Lunar  surface  dust  will  remain  a 
challenge  for  future  missions  to  the  Moon,  as  well  as  possible  landings  on  Mars. 

Mitigating  These  Challenges 

Each  of  these  challenges  of  vertical  flight  can  affect  the  pilot’s  cognitive  state  by  forc¬ 
ing  the  pilot  to  incur  additional,  mentally  demanding  processes  to  maintain  control 
and  stability  of  the  aircraft  or  spacecraft.  These  processes  require  maintaining  sig¬ 
nificant  amounts  of  information  in  short-term  memory  stores  and  mentally  relating 
and  projecting  disparate  variables  to  gain  a  clear  understanding  of  the  current  situ¬ 
ation  or  state.  As  Figure  1-5  illustrates,  the  many  challenges  that  pilots  face  can  be 
overwhelming. 


Figure  1-5:  Vertical  Flight  Challenges 


These  challenges  are  primarily  caused  by  a  loss  of  sensory  inputs,  or  a  loss  of  sta- 
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bility  that  requires  precision  control  to  avoid  or  recover  from  these  unsafe  conditions. 
The  loss  of  visual  inputs  and  the  need  for  precision  control  can  both  be  addressed 
through  the  use  of  displays  and  individual  display  design.  Each  of  these  vertical  flight 
challenges  can  be  mitigated  through  display  design  in  the  following  manner. 

•  Ground  Effect  While  the  presence  of  or  lack  of  ground  effect  presents  different 
challenges  for  different  V/STOL  aircraft,  the  challenges  all  relate  to  the  control 
of  vertical  speed  at  very  specific  altitudes.  Therefore,  the  integration  of  these 
two  sources  of  information  on  the  display  in  such  a  way  that  both  can  be  directly 
perceived  and  appropriately  manipulated  would  help  mitigate  this  challenge. 

•  Vortex  Ring  State:  As  vortex  rings  state  or  “settling  with  power”  is  caused  by 
an  excessive  descent  rate  at  low  airspeeds  while  applying  power,  the  display  of 
this  vertical  speed  limit  that  visually  indicates  the  current  position  in  relation 
to  that  limit  may  help  mitigate  this  challenge. 

•  Meteorological  Flying  Conditions:  Poor  meteorological  flying  conditions  that  re¬ 
quire  instrument  flight  rules  necessitate  that  critical  information  that  is  usually 
provided  by  outside  visual  cues,  be  displayed  in  an  intuitive  manner  inside  the 
cockpit. 

•  Visual  Disruptions:  Visual  disruptions,  which  also  temporarily  eliminate  many 
of  the  commonly  used  outside  visual  cues,  require  an  intuitive  display  of  critical 
flight  information  in  a  way  that  can  be  quickly  referenced.  This  way,  the  display 
can  be  used  in  conjunction  with  the  reduced  visual  cues  to  provide  a  clear 
understanding  of  the  situation. 

1.1.4  Problem  Statement 

The  challenging  conditions  that  accompany  vertical  flight,  in  particular  the  hovering 
and  landing  aspects  of  vertical  flight,  place  high  cognitive  demands  on  pilots.  Flight 
instrument  displays  are  becoming  increasingly  important  for  mitigating  these  chal¬ 
lenges  as  the  demand  for  vertical  aircraft  increases,  particularly  in  difficult  operating 


24 


conditions.  The  high  cognitive  demands  of  vertical  flight,  particularly  in  unfavorable 
operating  conditions,  demand  intuitive  and  perception-based  displays  that  present  a 
large  amount  of  information  to  the  pilot  quickly.  A  decision  support  tool  which  sat¬ 
isfies  this  need  by  taking  advantage  of  direct-perception  interaction  [17,  18]  through 
leveraging  ecological  perception  and  emergent  features  will  be  the  focus  of  this  re¬ 
search  effort. 


1.2  Research  Objectives 

In  order  to  address  the  problem  statement,  the  overarching  goal  of  this  research  is  to 
develop  a  decision  support  instrument  for  vertical  landing.  This  goal  will  be  addressed 
through  the  following  research  objectives: 

•  Objective  1.  Study  the  cognitive  strategies  employed  by  air  and  space¬ 
craft  pilots  conducting  vertical  descents.  In  order  to  achieve  this  objective, 
a  cognitive  task  analysis  was  conducted  and  a  cognitive  model  developed  for  ver¬ 
tical  landing  and  hover  operations,  both  of  which  are  described  in  Chapter  3. 
Based  on  the  data  collected,  the  propagation  of  external  and  internal  cognitive 
inputs  through  a  pilot’s  or  astronaut’s  cognitive  processes  and  the  mental  steps 
required  to  achieve  the  goals  of  the  mission  were  determined.  The  results  of 
this  analysis  and  model  are  outlined  as  design  requirements  for  vertical  land¬ 
ing  instruments.  A  literature  review  on  current  vertical  displays  and  the  way 
in  which  they  are  used  by  the  respective  aircraft  or  spacecraft  pilots  was  also 
conducted  as  part  of  this  analysis. 

•  Objective  2.  Develop  a  vertical  precision  landing  aid  for  use  in  aviation 
and  space  domains.  Based  on  the  results  of  objective  1,  an  integrated  flight 
instrument  display  component  that  addresses  the  design  requirements  outlined 
in  Chapter  3  was  designed  (described  in  Chapter  4).  A  discussion  of  the  design 
principles  applicable  to  this  display  component  and  integration  of  this  instru¬ 
ment  into  real-world  flight  systems  is  also  included  in  Chapter  3. 
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•  Objective  3.  Evaluate  the  effectiveness  of  the  new  vertical  precision 
landing  aid  on  hover  and  vertical  landing  performance.  To  address  this 
objective,  human-participant  experimentation  of  the  proposed  vertical  precision 
landing  aid  was  conducted.  A  description  of  the  experiment  is  outlined  in 
Chapter  5,  while  the  results  of  the  experiment  and  a  discussion  of  their  meaning 
are  found  in  Chapters  6  and  7. 

1.3  Thesis  Organization 

This  thesis  is  organized  into  the  following  chapters: 

•  Chapter  1,  Introduction,  introduces  and  describes  the  motivation  and  research 
objectives  of  this  thesis. 

•  Chapter  2,  Background,  provides  a  summary  of  the  current  technological  state 
of  air  and  spacecraft  flight  displays,  discusses  current  vertical  landing  flight 
display  research,  and  frames  the  context  of  the  research  objectives  introduced 
in  Chapter  1. 

•  Chapter  3,  Cognitive  Processes  for  Vertical  Landing  and  Hover  Operations,  out¬ 
lines  the  cognitive  task  analysis  approach  conducted  for  this  research.  A  model 
of  the  cognitive  strategies  employed  during  vertical  landings  is  introduced  and 
described  with  respect  to  the  cognitive  task  analysis  results. 

•  Chapter  4,  Vertical  Altitude  and  Velocity  Indicator,  describes  an  integrated 
flight  instrument  display  designed  to  aid  with  vertical  landings,  and  introduces 
the  design  rationale. 

•  Chapter  5,  Human  Participant  Experimentation,  discusses  the  predicted  per¬ 
formance  of  the  Vertical  Altitude  and  Velocity  Indicator  (VAVI).  Details  about 
the  objectives,  participants,  and  procedures  utilized  in  the  human  participant 
experimentation  of  the  VAVI  are  outlined. 
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•  Chapter  6,  Results,  presents  the  statistical  results  of  the  experiment  described 
in  the  Human  Participant  Experimentation  chapter. 

•  Chapter  7,  Discussion,  compares  the  results  of  the  human  participant  experi¬ 
ment  with  the  hypotheses  and  discusses  the  applicability  of  the  results  to  future 
integrated  flight  instrument  display  design. 

•  Chapter  8,  Conclusion,  summarizes  the  motivation  and  objectives  of  this  re¬ 
search,  how  well  the  objectives  were  met,  and  the  key  contributions.  Suggestions 
for  future  work  are  also  provided. 
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Chapter  2 


Background 


This  chapter  presents  a  chronological  summary  of  air  and  spacecraft  displays  that 
either  currently  exist  or  have  existed  to  aid  in  vertical  and/or  precision  operations 
for  helicopters  and  V/STOL  aircraft.  First,  displays  used  for  Lunar  descent  will  be 
discussed.  Then,  displays  currently  used  for  aircraft  will  be  described,  followed  by 
rotorcraft  and  V /STOL  operations.  Finally,  a  summary  of  related  research  on  vertical 
descent  displays  will  be  presented. 


2.1  Apollo  Landing  Display  Systems 

The  Apollo  Lunar  Module  (LM)  human-system  interface  is  characteristic  of  the  1960’s 
era  of  flight  deck  design.  The  LM  cockpit  consisted  primarily  of  mechanical  instru¬ 
ments  such  as  pressure  gauges,  gyros,  and  switches  as  depicted  in  Figure  2-1.  Since 
the  United  States  has  not  landed  humans  on  the  Moon  or  other  planets  since  1972, 
these  Apollo-era  spacecraft  cockpit  displays  remain  the  most  current  Lunar  landing 
technology. 

The  lack  of  cathode  ray  tube  (CRT)  or  liquid  crystal  displays  (LCD)  resulted  in 
segregated  information.  The  lack  of  automation  and  advanced  glass  cockpit  technol¬ 
ogy  that  exists  today,  left  the  mentally  demanding  tasks  of  integrating  information 
and  drawing  conclusions  about  the  vehicle  state  up  to  the  crew.  Few  instruments  and 
tools  directly  supported  the  crew  in  their  critical  monitoring  and  commanding  tasks. 
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The  vertical  descent  portion  of  the  Lunar  landing  was  a  particularly  cognitively  de¬ 
manding  task,  jeopardizing  the  accuracy  and  safety  of  a  landing.  The  few  vertical 
landing  aids  to  help  minimize  the  cognitive  demand  are  discussed  in  the  following 
section. 


Figure  2-1:  Apollo  Lunar  Module  Cockpit  (from  [19]) 


2.1.1  Apollo  Vertical  Landing  Aids 

During  the  vertical  descent  portion  of  the  Apollo  Lunar  landings,  attitude,  altitude, 
and  sink  rate  information  as  well  as  any  translational  rates  were  the  most  critical 
pieces  of  information.  The  instruments  used  to  monitor  these  key  parameters,  though 
physically  located  close  together,  were  separate  instruments  that  had  to  be  scanned 
and  cognitively  integrated  together  by  the  crew  to  paint  a  complete  picture  of  the 
current  state  of  the  vehicle  and  the  landing.  Altitude  and  sink  rate  (called  altitude 
rate)  were  simple  tape  meter  instruments  co-located  below  the  translational  infor¬ 
mation  and  adjacent  to  the  flight  director  attitude  indicator  (FDAI)  (referred  to  as 
the  “8-ball”)  illustrated  in  Figure  2-2a.  In  addition,  the  data  storage  and  keyboard 
(DSKY)  illustrated  in  Figure  2-2b  was  the  display  that  enabled  the  crew  to  com¬ 
municate  with  the  Apollo  guidance  computer  and  was  therefore  also  an  important 
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informational  tool.  Through  the  use  of  “noun”  and  “verb”  commands,  astronauts 
could  get  rate,  direction,  and  time  information  and  give  commands  to  the  guidance 
computer.  For  example,  the  mission  commander  typed  in  “Verb  50,  Noun  18”  during 
a  Lunar  landing  to  tell  the  computer  to  maneuver  to  the  proper  altitude  for  powered 
descent  initiation  [16]. 


(a)  Flight  Director  Attitude  Indicator  (” 8-Ball”) 


(b)  Data  Storage  and  Keyboard 


Figure  2-2:  Apollo  Lunar  Module  Cockpit  Instruments(from  [19]) 


2.2  Aircraft  Display  Systems 

From  the  time  of  the  Wright  brothers  and  no  display  systems  to  the  electro-optical 
instruments  used  today,  advances  in  aircraft  display  systems  have  been  tremendous 
over  the  last  century.  The  majority  of  those  advances  have  taken  place  in  the  last  three 
decades  advancing  from  purely  mechanical  instruments  to  the  electromechanical  era 
of  Apollo  and  on  to  the  electro-optical  instruments  widely  used  today.  Flight  displays 
are  a  critical  air  or  spacecraft  component  because  they  provide  necessary  vehicle  state 
information  in  the  absence  of  visual  cues  or  faulty  vestibular  feedback.  The  following 
section  will  be  a  short  description  of  the  most  commonly  employed  electro-optical 
display  system  seen  today  in  helicopters  and  V/STOL  aircraft. 


31 


2.2.1  Head-Up  Displays  (HUD) 


Head  up  displays  (HUD)  have  been  in  existence  since  before  1970s,  however  it  has 
only  been  in  the  last  decade  or  so  that  HUDs  elevated  to  the  category  of  primary 
flight  instruments  [20].  The  key  attribute  of  a  head  up  display  is  the  collimated 
image  of  flight  symbology  or  sensor  video  onto  a  piece  of  semi-reflecting  glass  such 
that  important  pieces  of  information  are  overlaid  directly  on  the  outside  view  of  the 
world  [20].  Though  originally  developed  for  weapons-aiming,  the  HUD  has  progressed 
into  a  widely  used  primary  flight  instrument  [20] ,  primarily  in  military  aircraft  where 
pilots  are  performing  many  tasks  at  once.  HUDs  eliminate  the  need  for  visual  scanning 
and  refocusing  of  the  eyes  from  the  electromechanical  instruments  in  the  cockpit  to  the 
outside  view.  Therefore  the  projection  of  HUD  flight  information  further  integrates 
data  to  include  outside  information  as  well  as  flight  instrument  data.  Specifically, 
the  display  of  altitude,  vertical  speed,  attitude,  and  heading  are  the  most  commonly 
monitored  flight  parameters  during  a  vertical  landing.  HUDs  are  a  particularly  useful 
technology  for  vertical  landings  during  which  the  outside  visual  cues  are  especially 
critical.  Figure  2-3  illustrates  an  example  of  a  head  up  display. 


Figure  2-3:  Head-Up  Display  (from  [21]) 
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2.3  Vertical  Aircraft  Landing  Aids 


Numerous  examples  of  vertical  landing  flight  displays  exist  today,  each  with  slight 
aesthetic  differences,  but  similar  key  pieces  of  information.  Currently  employed  flight 
displays  in  helicopters  and  V/STOL  aircraft  present  critical  flight  information  as 
disparate  elements  within  the  display.  In  this  section,  examples  of  cockpit  display 
elements  that  relate  to  altitude,  vertical  speed,  and  attitude  for  the  following  aircraft 
will  be  introduced  and  briefly  described.  These  aircraft  were  chosen  demonstrate  a 
cross-section  of  helicopters  and  V/STOL  aircraft. 

•  UH-60  Black  Hawk,  US  Army  (Figure  2-4a) 

•  AV-8B  Harrier,  US  Marine  Corps  (Figure  2-4b) 

•  MV-22  Osprey,  US  Marine  Corps  (Figure  2-4c) 


2.3.1  UH-60  Vertical  Descent  and  Hover  Displays 

The  UH-60  Black  Hawk  (Figure  2-4a)  is  a  front-line  utility  helicopter  used  by  the  US 
Army  for  air  assault,  air  calvary,  and  aeromedical  evacuation.  It  has  a  53-foot  rotor 
diameter  and  can  carry  11  combat-loaded  troops  [22], 

The  details  of  the  UH-60  HUD  are  depicted  in  Figure  2-5.  The  key  pieces  of 
information  which  relate  to  fundamental  vertical  landing  information  are  outlined  in 
Table  2.1  and  corresponds  to  the  numbers  in  Figure  2-5. 


Table  2.1:  UH-60  HUD  Description 


Number 

Definition 

7 

Barometric  Altitude  (MSL) 

10 

Velocity  Vector 

11 

Rate  of  Climb  Pointer 

12 

Radar  Altitude  (AGL)  -  Numeric 

13 

Minimum  Altitude  Warning 

14 

Radar  Altitude  (AGL)  -  Analog  Bar 

15 

AGL,  Vertical  Speed  Scale 
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In  this  HUD,  radar  altitude1  and  vertical  speed  are  depicted  along  the  same  analog 
scale,  while  the  current  radar  altitude  is  also  displayed  digitally  (up  to  1000  feet). 
The  radar  altitude  analog  bar  is  depicted  for  0-250  feet.  It  disappears  at  altitudes 
greater  than  250  feet  and  reappears  at  230  feet.  Barometric  altitude2  is  displayed 
digitally  in  the  upper  right  corner  of  the  display.  Likewise,  the  vertical  speed  scale 
will  indicate  vertical  speeds  up  to  +/-  2000  feet-per-minute  for  altitudes  up  to  200 
feet. 

The  velocity  vector  is  applicable  for  0-15  knots  at  which  point  it  disappears  for 
high  speeds.  Like  many  HUDs,  the  pilot  can  customize  the  display  to  include  some  or 
all  of  the  display  elements.  In  an  interview  with  a  Black  Hawk  pilot,  he  reported  that 
he  never  includes  the  analog  altitude  and  vertical  speed  scale  in  his  customization  of 
the  HUD  because  he  felt  that  it  cluttered  the  display.  This  indicates  that  he  does  not 
find  the  information  to  be  useful  in  his  tasks,  even  at  night  when  using  night  vision 
goggles  (NVG)  at  which  time  visual  cues  are  less  prominent. 

2.3.2  Harrier  Vertical  Descent  and  Hover  Displays 

The  AV-8B  Harrier  is  a  jet-lift  V /STOL  aircraft  that  uses  vectored-thrust  to  perform 
V/STOL  operations  at  low  speeds.  The  jet  exhaust  of  the  engine  is  directed  downward 
using  rotating  nozzles  which  allows  the  aircraft  to  counter  gravity.  The  Harrier  is  used 
by  the  US  Marine  Corps  for  close  air  support,  anti-air  warfare,  and  reconnaissance. 
It  is  capable  of  operating  from  carriers  and  remote  tactical  sites  [22] . 

The  Harrier  V/STOL  mode  HUD  is  illustrated  in  Figure  2-6  which  outlines  key 
features  of  the  display. 

In  the  Harrier  display,  vertical  speed  and  altitude  are  depicted  separately.  In  the 

internal  cockpit  display  (not  shown),  each  parameter  has  its  own  analog  scale  along 

which  the  current  value  is  indicated,  either  digitally  or  by  way  of  a  marker.  In  the 

HUD,  altitude  is  only  a  digital  readout  while  vertical  speed  is  both  a  digital  and 

1  radar  altitude  is  altitude  determined  by  a  radar-type  altimeter  and  is  the  actual  distance  from 
the  nearest  terrain  feature  directly  below  the  aircraft 

2barometric  altitude  is  altitude  determined  by  pressure  level  and  calculated  according  to  standard 
atmosphere  laws 
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Figure  2-6:  Harrier  HUD  (from  [24]) 


analog  depiction. 

The  Harrier  HUD  also  includes  two  unique  tools  both  for  normal  flight  and  ver¬ 
tical  landings  that  are  specific  to  the  Harrier’s  configuration.  First,  the  depressed 
attitude  symbol  outlined  in  Figure  2-6  is  fixed  8  degrees  below  the  waterline,  which 
represents  an  imaginary  line  going  through  the  middle  of  the  aircraft  in  the  vertical 
axis.  Therefore,  when  the  depressed  aircraft  symbol  is  on  the  horizon,  the  Harrier 
has  a  slight  nose-up  landing  attitude.  This  is  critical  for  a  soft  landing  of  the  Harrier 
because  it  is  designed  such  that  on  level  ground,  the  nose  is  situated  higher  than  the 
tail.  This  tool  eliminates  the  need  to  compensate  for  the  natural  irregular  Harrier  at¬ 
titude.  Second,  the  position  of  the  vertical  flight  path  symbol  (also  called  the  velocity 
vector)  gives  vertical  speed  information.  If  the  symbol  is  located  above  the  horizon, 
it  indicates  that  the  vehicle  is  climbing,  while  a  symbol  below  the  horizon  indicates  a 
descent.  At  less  than  about  60  knots  airspeed,  this  symbol  matches  the  vertical  rate. 
Therefore,  if  the  depressed  attitude  symbol  is  on  the  horizon,  and  the  vertical  flight 
path  symbol  is  3  degrees  below,  then  the  vehicle  is  descending  at  300  feet  per  minute. 
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2.3.3  Osprey  Vertical  Descent  and  Hover  Displays 


The  V-22  Osprey  is  a  tilt-rotor  aircraft  designed  for  use  by  US  Special  Operations 
Forces.  The  Osprey  is  capable  of  landing  like  a  helicopter  and  then  transitioning 
the  engine  nacelles  to  fly  like  a  turboprop  airplane  at  high  altitudes  and  speeds.  It’s 
capable  of  carrying  24  combat  troops  or  20,000  pounds  of  cargo  for  use  in  amphibious 
assault,  combat  support,  transport,  and  search  and  rescue  operations  [22], 

The  Osprey  head  down  hover  display  and  HUD  are  illustrated  in  Figures  2-7  and 
2-8  along  with  Tables  2.2  and  2.3  that  outline  key  features  of  both  displays  respec¬ 
tively.  Figure  2-8a  illustrates  the  HUD  for  hover  mode  and  Figure  2-8b  highlights 
key  features  of  the  HUD  hover  display. 


Figure  2-7:  V-22  Osprey  Display  (from  [25]) 


The  display  of  altitude  and  vertical  speed  on  both  displays  remains  consistent  or 
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Table  2.2:  V-22  Osprey  HDD  Description 


Number 

Definition 

1 

Groundspeed 

2 

Velocity  Vector  Scale 

3 

Acceleration  Cue 

4 

Velocity  Vector 

5 

Commanded  Radar  Altitude  Pointer 

6 

Radar  Altitude  (AGL) 

7 

Vertical  Speed  Range 

8 

Radar  Altitude  Low  Set  Pointer 
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Figure  2-8;  MV-22  Osprey  Hover  Mode  HUD  (from  [25]) 


Table  2.3:  V-22  Osprey  HUD  Description 


Number 

Definition 

14 

Acceleration  Cue 

15 

Velocity  Vector  (Hover) 

16 

Digital  Barometric  Altimeter 

17 

Vertical  Velocity 

18 

Digital  Radar  Altimeter 

19 

Master  Alert 
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at  least  similar  to  other  rotorcraft  or  V/STOL  aircraft.  However,  in  this  HUD,  the 
digital  altitude  is  located  just  to  the  left  of  the  vertical  velocity  scale.  This  was  done 
in  an  attempt  to  place  the  two  pieces  of  information  close  together  in  such  a  way  that 
the  pilot  can  more  clearly  decipher  and  combine  the  two  pieces  of  information  to  help 
gain  a  clear  understanding  of  the  vehicle  state.  However,  the  digital  altitude  that 
is  displayed  close  to  the  vertical  speed  scale  is  barometric  altitude  and  not  a  radar 
altitude.  Radar  altitude,  which  is  displayed  in  the  lower  right  corner,  would  be  the 
more  useful  altitude  at  the  low  altitudes  at  which  hover  and  landing  operations  take 
place. 


2.4  Related  Research  in  Vertical  Displays 

A  review  of  the  literature  indicates  that  the  majority  of  related  research  on  display 
concepts  for  approach  and  landing  of  helicopters  or  V/STOL  aircraft  does  not  ade¬ 
quately  address  the  vertical  descent  and  hover  tasks  in  the  vertical  situation  specif¬ 
ically.  While  many  of  the  proposed  display  designs  incorporate  creative  and  unique 
ways  to  present  information  in  both  the  horizontal  and  vertical  planes,  the  majority 
of  the  designs  display  disassociated  vertical  speed  and  altitude  information. 

Three  proposed  display  concepts  that  did  not  become  operational  are  illustrated 
in  Figure  2-9.  Figure  2-9a  and  b  illustrate  two  proposed  designs  that  present  the 
necessary  vertical  situation  information  not  only  as  disassociated  information,  but  on 
opposite  sides  of  the  primary  center  display.  Altitude,  vertical  speed,  and  airspeed 
are  all  displayed  as  fixed  scales  with  moving  pointers.  Figure  2-9c  also  illustrates 
a  proposed  V/STOL  display  which  includes  disparate  information  in  two  different 
analog  forms.  In  this  display,  to  address  the  vertical  situation,  altitude  and  ground 
speed  are  displayed  along  the  left  and  right  sides  respectively  as  fixed  analog  scales 
somewhat  combined  with  the  center  display.  Vertical  speed,  however,  is  displayed  to 
the  left  as  a  totally  disjointed  instrument.  For  all  three  displays,  this  critical  approach 
and  landing  information  is  presented  as  raw  data  that  that  must  be  extracted  and 
comprehended  in  relation  to  the  pilot’s  specific  tasks  and  goals.  Therefore,  these 
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Airspeed 


displays  provide  no  integration  of  the  vertical  situation  parameters  such  as  altitude 
and  vertical  speed  for  direct  perception  of  the  relevant  meaning  of  the  values.  In  this 
way,  neither  of  these  proposed  displays  specifically  provides  decision  support  for  the 
vertical  descent  portion  of  flight. 


Altitude 

Vertical 

Speed 


Sink  Rate 
Ribbon 


Relative 
Vertical  Speed 


Figure  2-10:  Display  for  Precision  Hover  (from  [27]) 

One  of  a  series  of  displays  developed  and  tested  using  the  NASA  Ames  V/STOL 
System  Research  Aircraft  [27,  7,  28]  illustrated  in  Figure  2-10  does  attempt  to  inte¬ 
grate  the  vertical  situation  information.  In  this  display,  commanded  vertical  velocity 
is  depicted  by  the  diamond  located  along  the  allowable  sink  rate  ribbon.  The  sink  rate 
ribbon  indicates  the  allowable  range  of  sink  rate  which  will  ensure  a  safe  descent,  but 
does  not  visually  map  directly  to  any  vertical  speed  values.  Therefore  it  only  provides 
a  relative  vertical  velocity  indication,  while  the  exact  vertical  speed  that  corresponds 
to  the  diamond  is  displayed  digitally  at  the  top  of  the  display.  Similarly,  a  relative 
altitude  indication  is  visually  displayed  as  the  distance  between  the  “deck  bar”  and 
aircraft  symbol,  however  the  only  exact  information  is  displayed  in  the  upper  left 
corner. 

While  the  display  of  allowable  sink  rate  limits  in  Figure  2-10  is  very  useful  to 
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the  pilot,  the  physical  separation  of  this  relative  information  and  the  exact  value 
requires  the  pilot  to  move  his/her  gaze  in  order  to  get  the  full  suite  of  information 
corresponding  to  vertical  speed.  The  same  is  true  for  altitude  which  introduces  an 
extra  step  by  requiring  the  pilot  to  mentally  store  information  from  one  part  of  the 
display  while  gazing  at  another  part  of  the  display.  This  is  especially  cumbersome 
during  high  workload  descents  during  which  altitude  and  vertical  speed  information 
is  critical.  In  addition,  by  placing  the  digital  vertical  speed  and  altitude  values  right 
on  top  of  each  other  in  the  upper  right  corner,  the  pilot  must  not  only  search  for  that 
general  area  of  the  display,  but  also  search  between  those  numbers  for  the  number  of 
interest,  making  it  error-prone  and  slow.  Numerous  other  display  concepts  took  the 
same  approach  of  physically  separating  relative  and  exact  representations  of  vertical 
situation  information  (see  references  [4,  2]  for  more  examples). 

Another  disadvantage  of  the  displays  illustrated  in  Figures  2-9  and  2-10  is  the 
lack  of  goal-relevant  vehicle  state  information  that  can  be  quickly  assimilated  with 
a  single  glance.  For  the  display  in  Figure  2-10  this  is  attributed  to  the  interactive 
nature  of  the  display.  This  display  requires  the  pilot  to  “move  the  velocity  predictor 
ball  to  the  landing  pad  and  then  follow  the  pad  as  it  approaches  and  converges 
on  the  aircraft  reference  symbol”  [27,  p.166].  As  mentioned  in  Chapter  1,  most 
helicopter  and  V/STOL  pilots  are  extremely  dependent  on  their  visual  inputs  when 
flying,  especially  during  vertical  operations.  Therefore,  only  in  the  total  absence  of 
all  outside  visual  cues  (which  would  require  an  extreme  situation),  would  a  pilot  be 
willing  to  totally  remove  his/her  visual  attention  from  the  outside  world  long  enough 
to  focus  on,  comprehend,  and  control  multiple  symbols  within  a  display  in  order  to 
control  his/her  aircraft.  Displays  such  as  those  in  Figures  2-9  and  2-10,  require  too 
much  information  to  be  perceived  and  comprehended  too  quickly  to  be  useful  during 
a  stressful  vertical  landing. 
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Chapter  3 


Cognitive  Processes  for  Vertical 
Landing  and  Hover  Operations 


This  chapter  provides  an  analysis  of  the  cognitive  strategies  employed  by  air  and 
spacecraft  pilots  during  vertical  landings.  It  then  summarizes  the  design  requirements 
that  were  revealed  through  the  analysis  for  use  in  the  design  of  future  displays  or  flight 
symbology. 


3.1  Cognitive  Task  Analysis 

A  cognitive  task  analysis  was  performed  to  gain  a  better  understanding  of  air  and 
spacecraft  pilots’  cognitive  processes,  key  challenges,  and  what  flight  parameters  are 
most  crucial  during  a  vertical  landing.  A  cognitive  task  analysis,  as  defined  by  Chip- 
man,  Schraagen,  and  Shalin  is  “the  extension  of  traditional  task  analysis  techniques 
to  yield  information  about  the  knowledge,  thought  processes,  and  goal  structures  that 
underlie  observable  task  performance”  [29,  p.3].  The  cognitive  task  analysis  used  in 
this  thesis  corresponds  to  this  definition  and  included  the  following  techniques: 

•  Semi- Structured  Interviews:  Interviews  were  conducted  with  various  helicopter, 
Harrier,  and  Osprey  pilots,  Apollo  engineers  and  ground  controller  personnel, 
and  the  following  Apollo  astronauts:  Buzz  Aldrin,  John  Young,  and  Harrison 
Schmitt.  Open-ended  questions  were  used  to  guide  the  interviews.  Interviews 
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were  either  conducted  in  person  or  over  the  telephone  and  each  interview  lasted 
approximately  45  minutes  to  1  hour. 

•  Simulator  Experience :  An  AV-8B  Harrier  pilot  located  at  Cherry  Point  Marine 
Air  Station  in  Cherry  Point  NC  provided  simulator  demonstrations  and  the 
opportunity  to  fly  the  simulator.  A  V-22  Osprey  flight  instructor  at  New  River 
Marine  Air  Station  in  Jacksonville,  NC  provided  a  two  hour  tour  of  the  six 
degree  of  freedom  Osprey  simulator. 

•  Transcript  Analysis:  Analysis  of  the  Apollo  Lunar  Surface  Journal  was  con¬ 
ducted  to  better  understand,  through  the  astronaut’s  communications,  how  a 
vertical  landing  on  the  Moon  was  conducted. 

•  Storyboards  and  Timelines:  A  compilation  of  the  transcript  analysis  and  other 
resources  were  used  to  create  a  storyboard  and  timeline  of  the  Apollo  Lunar 
landing  which  highlighted  key  tasks  and  periods  of  high  workload. 

3.2  A  Cognitive  Model  for  Vertical  Landing 

Endsley’s  model  of  situation  awareness  (SA)  was  modified  to  illustrate  the  cognitive 
processes  that  are  required  for  achieving  SA  for  pilots  or  astronauts  performing  verti¬ 
cal  landings  or  hover  operations  [30].  This  model,  illustrated  in  Figure  3-1,  captures 
the  key  elements  of  SA  as  well  as  the  external  information  and  information  stored 
in  memory  necessary  for  developing  SA  during  vertical  landings.  Everything  con¬ 
tained  within  the  gray  box  represents  the  cognitive  processes  of  a  pilot  or  astronaut, 
while  everything  outside  of  that  box  represents  the  external  world.  Situation  aware¬ 
ness,  as  defined  by  Endsley,  is  “the  perception  of  the  elements  in  the  environment 
within  a  volume  of  time  and  space,  the  comprehension  of  their  meaning,  and  the 
projection  of  their  status  in  the  near  future”  [30,  p.36].  Situation  awareness  flows 
directly  into  decision-making  and  action  implementation  at  which  point  the  effects 
are  perceived  through  the  feedback  and  SA  is  reconstructed  and  reevaluated.  This 
cascading  sequence  requires  high  situation  awareness  for  making  a  good  decision  and 
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corresponding  action.  Each  level  of  SA  and  the  surrounding  cognitive  processes  for 
managing  altitude  and  vertical  speed  during  a  vertical  landing  are  described  in  more 
detail  in  the  following  subsections. 
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3.2.1  External  Inputs 

The  external  inputs  highlighted  in  the  model  emphasize  the  primary  sources  of  in¬ 
formation  the  pilots  rely  on  to  determine  their  current  position  in  relation  to  their 
goals  [31].  The  three  input  sources  include: 

•  Visual:  information  provided  by  the  outside  view  of  the  world. 

•  Vestibular  System:  information  provided  by  changes  in  gravitational  forces  on 
the  body. 

•  Symbolic:  information  provided  by  flight  instruments  and  displays  within  the 
cockpit. 

The  visual  inputs  specific  to  vertical  operations  include  such  things  as  geographic 
and  man-made  landmarks,  shadows,  dust,  and  the  horizon.  Although  many  heli¬ 
copters  and  V/STOL  aircraft  are  equipped  with  old  navigational  technology  such 
as  Lorans  and  VHF-Omnidirectional-Range  (VOR)  radio  navigation,  or  newer  tech¬ 
nology  such  as  GPS,  some  operations  heavily  utilize  pilotage1.  Similarly  during  all 
Apollo  landings,  predetermined  landmarks  were  used  to  ensure  the  spacecraft  was  on 
the  appropriate  trajectory  for  landing.  Landmarks  also  provide  key  peripheral  and 
depth  perception  cues  which  indicate  vehicle  height.  The  familiarity  with  the  objects 
in  conjunction  with  the  visual  feedback  they  provide  in  terms  of  texture  and  detail 
provides  a  sense  of  vehicle  position  relative  to  the  ground  to  ensure  current  and  future 
ground  clearance.  Pilots  and  astronauts  also  use  the  placement  of  landmarks  or  the 
horizon  within  their  windows  to  determine  any  lateral  or  longitudinal  movement  and 
descent  rates  for  positioning  during  a  stationary  hover.  A  Harrier  pilot  described 
how  he  would  situate  the  aircraft  such  that  he  had  a  landmark  to  the  front  and  to 
the  side  of  him  in  order  to  provide  both  vertical  and  horizontal  relative  movement 
information.  Shadows  (if  the  operation  is  conducted  at  the  appropriate  time)  can  also 
provide  ground  clearance  information  through  their  dynamic  size  during  a  descent, 

1  Pilotage  is  navigation  by  visual  reference  to  landmarks  [32] 
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as  can  dust  which  indicates  a  certain  proximity  of  the  thrusters  or  rotor  blades  to  the 
ground. 

The  pilot’s  vestibular  system,  located  in  the  inner  ear,  responds  to  gravity  to  sense 
angular  motion,  accelerations,  and  position  [33].  In  vertical  operations,  this  system  is 
important  for  sensing  lateral,  vertical,  and  angular  motion  as  well  as  any  acceleration 
in  those  directions  in  order  to  maintain  a  hover  or  descent  with  minimal  lateral  or 
longitudinal  motion.  The  vestibular  system  can  also  be  misleading  or  unreliable.  Ac¬ 
cording  to  Young,  visual  information  “frequently  conflicts  with  motion  or  orientation 
signals  emanating  from  the  vestibular  system”  [33,  p.83]  and  some  experts  argue  that 
“pilots  should  ignore  vestibular  information  all  together  to  be  safe  in  a  low-altitude 
environment”  [31,  p.50]. 

The  symbolic  information,  or  the  presentation  of  that  information  within  the 
cockpit,  is  often  unique  to  the  vehicle  platform.  As  outlined  in  Figure  3-1,  an  altime¬ 
ter,  vertical  speed  indicator  (VSI),  attitude  direction  indicator  (ADI),  and  on-board 
charts  and  vehicle  status  warning  lights  are  just  a  few  of  the  flight  instruments  that 
provide  symbolic  information.  Since  visual  and  vestibular  feedback  can  provide  erro¬ 
neous  cues,  the  symbolic  information  displayed  through  these  flight  instruments  is  a 
critical  input  for  achieving  high  situation  awareness. 

3.2.2  Internal  Inputs 

Situation  awareness  is  also  driven  by  internal  cognitive  inputs  from  long  term  or 
working  memory.  Any  pre-acquired  knowledge  gained  through  training  or  experience 
is  of  key  importance  in  the  tasks  in  this  model.  This  source  of  information  is  stored 
in  long-term  memory  and  flows  directly  into  the  information-processing  mechanism. 
Long-term  memory  structures  allow  pilots  to  act  appropriately  based  on  scripts  that 
form  through  repeated  training  and  past  experiences  [30]. 

Working  memory,  not  explicitly  represented  in  the  model,  but  is  subsumed  in  the 
information-processing  function,  compliments  the  information-processing  mechanism 
by  filling  the  gap  between  the  long  term  memory  stores  and  the  unique  elements  of 
the  current  environment  in  order  to  determine  the  appropriate  action.  This  is  where 
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the  majority  of  active  processing  takes  place. 

In  addition  to  memory,  the  goals  of  the  mission  also  drive  the  cognitive  processes 
of  the  pilots.  In  the  case  of  vertical  precision  landing  and/or  hover  operations,  the 
two  foremost  goals  are  safety  and  mission  success  within  which  several  minor  goals 
reside.  Safety  is  dependent  on  many  factors,  however  those  that  correspond  directly  to 
vertical  landing  and  hovering  include  maintaining  a  safe  attitude  (i.e.  staying  within 
the  pitch  and  bank  constraints  of  the  vehicle),  safe  descent  rates,  and  an  obstacle-free 
landing  zone.  Mission  success  for  precision  landing  can  be  directly  measured  by  the 
accuracy  and  safety  of  the  operations  and  the  timeliness,  which  can  be  critical  in 
many  space  and  military  operations. 

Preattentive  processing,  also  not  explicitly  represented  in  the  model,  is  one  infor¬ 
mation  processing  mechanism  that  helps  bridge  the  gap  between  the  external  inputs 
and  SA.  According  to  Endsley  [30],  certain  object  characteristics,  such  as  spatial 
proximity,  color,  etc,  are  detected  initially  through  the  parallel  processing  of  envi¬ 
ronmental  features  through  preattentive  sensory  stores.  These  serve  as  the  cues  that 
drive  further  localized  attention  on  the  objects  that  are  the  most  salient,  to  achieve 
perception.  Therefore  cue  salience  has  a  significant  impact  on  which  elements  of  the 
environment  are  attended  to  and  subsequently  perceived  [30] . 

3.2.3  Perception 

The  first  level  of  SA  is  perception  of  the  “status,  attributes,  and  dynamics  of  relevant 
elements  in  the  environment”  [30,  p.36].  For  pilots  and  astronauts,  this  information 
comes  directly  from  the  three  primary  external  sources  described  in  section  3.2.1. 
While  some  of  the  attributes  can  be  directly  perceived  from  the  flight  instruments 
(such  as  altitude,  attitude,  and  vertical  speed),  other  characteristics  are  perceived 
from  the  outside  view  (e.g.  texture,  detail,  projection  of  light)  and  are  then  processed 
to  gain  additional  information.  These  are  the  processes  that  facilitate  the  transition 
to  comprehension  and  will  be  described  in  more  detail  in  the  next  section.  Information 
such  as  the  relative  location  of  ground  obstacles  is  more  easily  perceived  directly  from 
the  outside  world.  It  is  the  parallel  processing  of  these  various  sensory  external  inputs 
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that  creates  the  first  level  of  situation  awareness. 


3.2.4  Comprehension  and  Projection 

Comprehension  of  the  current  situation  is  the  second  level  of  situation  awareness. 
At  this  level,  the  integration  of  level  1  information  and  the  external  goals  creates 
an  understanding  of  those  perceived  elements  with  respect  to  the  relevant  goals  [30] . 
During  comprehension  the  pilots  compensate  for  missing  or  segregated  information 
by  relying  on  memory  and  information-processing  to  get  them  from  the  information 
they  can  directly  perceive  to  the  information  they  actually  need  to  achieve  their 
goals.  The  greater  the  mismatch  in  the  information  available  and  the  information 
needed  to  achieve  the  goals,  the  higher  the  cognitive  workload  required  to  perform 
information-processing  and  recall  from  memory. 

Some  of  the  key  goals  of  a  Lunar  or  earth-based  vertical  landing  are  outlined  in 
Figure  3-1  and  feed  directly  into  the  information-processing  mechanism.  The  first 
goal,  safety,  includes  maintaining  safe  sink  rates  and  control  of  the  vehicle  state,  and 
ensuring  a  clear  landing  site  and  safe  ground  clearance.  To  maintain  safe  sink  rates, 
the  pilot  or  astronaut  must  perceive  the  current  altitude  and  current  vertical  speed 
from  a  combination  of  individual  symbolic  displays,  vestibular  feedback,  and  visual 
stimulants.  The  perception  of  this  information  is  important,  but  the  comprehension  of 
its  meaning  is  arguably  more  critical.  While  the  outside  world  and  vestibular  system 
can  provide  relative  sensory  feedback  on  altitude  and  vertical  speed,  the  instruments 
within  the  cockpit  provide  more  accurate  and  precise  information  especially  in  the 
absence  of  visual  cues  during  the  low  visibility  situations  discussed  in  Chapter  1.  Since 
current  vertical  speed  indicators  do  not  indicate  sink  rate  constraints  for  safe  descents, 
the  comprehension  of  this  information  requires  that  the  pilots  or  astronauts  recall  from 
memory  the  heuristic  they  were  taught  that  guides  the  dynamic  safety  constraints 
relative  to  altitude.  The  practice  of  teaching  safe  descent  heuristics  is  a  common 
one  that  was  uncovered  through  the  cognitive  task  analysis.  They  either  apply  these 
heuristics  or  match  the  current  situation  to  a  past  experience  stored  in  memory  and 
perform  according  to  their  knowledge  of  those  results  (according  to  the  theory  of 
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naturalistic  decision-making  [34]).  In  interviews  conducted  with  Harrier,  Osprey,  and 
helicopter  pilots,  they  reported  this  error-prone  and  mentally  taxing  step,  usually 
referring  to  their  training  as  the  primary  source  of  information  for  comprehending  the 
raw  data  with  respect  to  the  safe  sink  rate  constraints.  However,  relying  on  memory 
for  the  heuristic  does  not  provide  a  comprehension  of  the  vehicle  state  relative  to  those 
constraints.  Consequently  an  additional  step  is  required,  which  involves  comparing 
the  dynamic  sink  rate  (which  may  only  involve  perceiving  a  digital  number  on  the 
display)  to  the  known  heuristic,  while  compensating  for  the  rapidly  changing  value. 
This  process  is  illustrated  in  Figure  3-2  and  demonstrates  the  mentally  taxing  process 
associated  with  comprehending  perceived  raw  data  in  a  dynamic  environment. 


Figure  3-2:  Progression  from  perception  to  comprehension  of  vehicle  state  relative  to 
safety  constraints 

Pilots  and  astronauts  are  also  concerned  with  vehicle  state  relative  to  the  state 
in  which  they  wish  to  be.  Hovering  is  a  good  example  of  this  situation.  In  order  to 
hover  at  a  precise  altitude,  pilots  must  focus  on  altitude,  vertical  speed,  heading,  and 
torque  (for  helicopters)  or  nozzle/rotor  position  and  power  (for  Harriers  and  Ospreys), 
switching  between  them  in  order  to  achieve  the  desired  state  through  the  appropriate 
control  inputs.  The  individual  pieces  of  information  may  be  directly  perceived  as 
values  on  the  display,  but  the  meaning  of  those  values  relative  to  the  goal  requires 
the  mental  integration  of  the  two  sources  of  information.  Helicopter  pilots  reported 
that  when  visual  cues  are  absent,  this  mental  integration  is  overwhelming  at  times. 

Ensuring  a  clear  landing  site  and  ground  clearance  during  a  descent  primarily  de¬ 
pends  on  the  pilot’s  comprehension  of  altitude  above  the  ground,  taking  into  account 
any  obstacles  or  hazards  below.  Shadows  and  dust  provide  visual  cues  of  relative 
altitude  which  pilots  utilize.  Like  sink  rate,  the  direct  perception  of  the  size  of  the 
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shadow  is  meaningless  without  pulling  from  long-term  memory  stores  to  comprehend 
the  approximate  altitude  associated  with  that  shadow  size.  While  Harrier,  Osprey, 
and  helicopter  pilots  reported  using  the  shadow  occasionally  as  a  cue,  they  did  not 
specifically  address  the  use  of  this  cue  in  training.  In  contrast,  Apollo  astronauts 
trained  to  use  the  shadow  as  a  relative  altitude  reference.  Since  the  Moon  landings 
happened  at  well-known  and  pre-planned  times,  the  shadow  size  was  predictable  and 
could  therefore  be  incorporated  into  training.  Figure  3-3  illustrates  the  shadow  of  the 
Apollo  11  Lunar  Module  on  descent.  Both  groups,  however,  must  pull  from  experience 
and  memory  to  comprehend  the  perceived  shadow  size  as  illustrated  in  Figure  3-1. 


Figure  3-3:  Apollo  11  Lunar  Module  Shadow 


Pilots  and  astronauts  also  depend  on  the  perception  of  values  on  their  displays  for 
altitude  information.  A  digitally  displayed  value  of  altitude  can  be  directly  perceived, 
but  to  comprehend  its  meaning  in  terms  of  ground  clearance  is  more  difficult.  Analog 
representations  of  altitude  allow  for  a  more  direct  perception  of  relative  altitude 
and  thus  vehicle  position  as  it  relates  to  the  goals.  Several  aircraft  displays  provide 
barometric  altitude  only  (with  knowledge  of  ground  altitude)  or  display  it  as  the 
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primary  altitude  of  interest  with  radar  altitude  displayed  in  a  secondary  location. 
Barometric  altitude  is  often  more  accurately  measurable  and  thus  is  the  primary 
reason  for  predominantly  displaying  it.  For  vertical  operations  close  to  the  ground, 
however,  barometric  altimeters  are  virtually  useless.  One  barometric  altimeter,  the 
Stratomaster  Maxi  Single  is  only  calibrated  to  ±30  ft,  which  at  low  altitudes  is 
not  very  helpful  [35].  Therefore,  ground  clearance  or  true  height  above  the  ground 
is  of  chief  importance  during  vertical  landings.  In  situations  in  which  this  critical 
information  is  not  displayed  such  that  it  must  be  visually  estimated,  ground  altitude 
and  barometric  altitude  must  be  mentally  integrated,  creating  a  mentally  taxing 
vertical  landing. 

To  address  the  timeliness  goal  if  landing  in  a  narrow  window  of  time  is  critical, 
as  it  was  during  the  Apollo  missions,  comprehending  the  time  to  touchdown  requires 
mentally  integrating  altitude  and  vertical  speed  (both  highly  transient)  at  any  given 
moment  in  time.  This  extremely  simple  task  for  automation  can  be  mentally  de¬ 
manding  step  for  a  human  during  the  high  stress  period  of  a  vertical  descent. 

The  third  and  highest  level  of  situation  awareness  is  the  projection  of  the  compre¬ 
hended  information  into  the  future.  Projection  is  achieved  when  an  understanding  of 
the  perceived  attributes  and  the  dynamics  of  the  specific  environment  can  be  extended 
into  an  understanding  of  a  future  state  relative  to  the  goals  [30] .  For  pilots  and  astro¬ 
nauts  performing  vertical  landings,  comprehension  and  projection  are  tightly  coupled 
due  to  the  highly  dynamic  nature  of  this  phase  of  flight.  Since  the  time  constants 
are  so  short  and  part  of  understanding  involves  projecting  into  the  future,  these  two 
steps  are  essentially  concurrent.  In  other  words,  since  it  is  a  manual  control  task 
during  which  flight  parameters  are  changing  every  second,  the  extent  of  the  pilot’s 
projection  of  vehicle  state  into  the  future  is  limited  because  it  is  happening  almost 
concurrently  with  the  comprehension. 

Some  of  the  information  that  the  pilots  are  able  to  project  further  into  the  fu¬ 
ture,  however,  are  the  approximate  touchdown  point  and  the  safety  of  that  site.  To 
determine  the  landing  site,  pilots  and  astronauts  must  combine  rate  and  position 
information  and  project  it  into  the  future.  Likewise,  determining  the  safety  of  a 
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projected  landing  site  involves  mentally  comparing  the  perceived  and  comprehended 
relative  position  of  obstacles  and  hazards,  to  the  projected  landing  site  in  order  assess 
the  safety  situation.  In  the  model  illustrated  in  Figure  3-1,  these  mental  steps  are 
highlighted  from  level  1  to  levels  2  and  3  SA. 

It  is  also  important  to  note  that  several  of  the  environmental  cues  that  are  often 
used,  perceived,  and  propagated  through  each  level  of  situation  awareness  are  unreli¬ 
able.  For  example,  the  use  of  familiar  landmarks  to  determine  location  is  a  common 
perception-comprehension  sequence  used  by  pilots  and  the  Apollo  astronauts.  A  mis¬ 
match  in  the  transition  from  perception  of  individual  landmarks  to  their  meaning 
about  vehicle  position  took  place  during  the  Apollo  15  mission.  Mission  commander 
Dave  Scott  revealed  that  right  after  pitchover  and  upon  his  first  view  of  the  landing 
site,  he  was  completely  disoriented.  As  a  result,  he  and  astronaut  James  Irwin  landed 
primarily  using  instrument  flight  rules  (IFR).  It  was  later  determined  that  this  confu¬ 
sion  resulted  from  an  exaggeration  of  the  landscape  in  the  Lunar  surface  model  used 
for  simulated  landings  [16].  Therefore  the  astronauts  perceived  the  outside  view,  but 
were  unable  to  comprehend  positional  information  from  it.  As  previously  discussed, 
dust  is  another  often  unreliable  cue  which  is  highly  dependent  on  external  factors 
such  as  the  surrounding  environment.  For  example,  sand  will  propagate  through  the 
environment  much  differently  than  dirt  or  snow.  The  perception  of  dust  kicked  up 
by  thrusters  or  rotor  blades  can  be  associated  with  an  altitude  which  corresponds  to 
past  experiences,  but  their  unreliable  nature  could  cause  trouble  if  it  is  not  correctly 
perceived.  These  two  examples  illustrate  how  an  error  in  one  level  of  situation  aware¬ 
ness  will  propagate  through  the  levels  causing  increased  workload  and  an  increased 
chance  of  serious  errors. 


3.3  Vertical  Landing  Display  Requirements 

The  cognitive  task  analysis  and  adaptation  of  the  cognitive  processing  model  revealed 
that  current  vertical  landing  displays  require  a  high  cognitive  workload  and  introduce 
room  for  error  by  requiring  pilots  to  mentally  integrate  and  perform  calculations. 
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More  specifically,  a  mismatch  between  the  data  provided  by  current  instruments  and 
the  information  needed  by  pilots  to  ensure  a  safe  and  precise  landing  became  apparent 
through  the  cognitive  processing  model.  Integrating  these  instruments  to  provide  a 
salient  display  of  the  information  required  in  a  form  that  matches  the  mental  model  of 
the  pilot  and  provides  for  direct-perception  interaction,  should  significantly  improve 
vertical  landing  operations.  The  design  requirements  identified  and  listed  below  for 
the  design  of  future  aircraft  and  spacecraft  vertical  landing  and  hover  displays  moti¬ 
vated  the  design  of  the  proposed  vertical  precision  landing  aid  introduced  in  the  next 
chapter. 

1.  Provide  obvious  display  of  sink  rate  safety  constraints  and  a  means  for  attain¬ 
ing  pilot  attention  when  the  constraints  are  violated.  The  display  of  maximum 
allowable  rates  of  descent  provides  an  indication  of  sink  rate  relative  to  the  rele¬ 
vant  goal  of  landing  safely.  Any  lag  associated  with  vertical  speed  measurement 
needs  to  be  eliminated  to  provide  reliable  information  [4]. 

2.  Enable  direct  perception  of  the  combination  of  vehicle  altitude  and  vertical  speed 
to  assist  in  quickly  determining  vehicle  hover  state.  The  combination  or  inte¬ 
gration  of  display  parameters  allows  several  parameters  to  be  observed  from 
one  element  [4]  which  addresses  the  difficulty  in  integrating  dynamic  vehicle 
parameters  to  gain  position  information. 

3.  Display  vehicle  altitude  above  the  ground  as  the  primary  displayed  altitude  dur¬ 
ing  vertical  operations.  This  height,  as  opposed  to  the  altitude  above  sea  level, 
is  of  primary  concern  during  a  descent.  An  analog  representation  of  altitude 
inherently  provides  a  rough  indication  of  the  rate  of  change  and  specifically 
addresses  the  difficulty  posed  by  a  loss  of  visual  cues  for  determining  sink  rate. 

4.  Display  the  approximate  time  to  touchdown.  The  integration  of  vertical  speed 
and  altitude  during  a  vertical  descent  should  be  done  by  the  automation  to 
remove  this  cognitively  demanding  step. 
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Chapter  4 


Vertical  Altitude  and  Velocity 
Indicator  (VAVI) 


This  chapter  presents  the  details  of  a  precision  landing  aid  called  the  vertical  altitude 
and  velocity  indicator  (VAVI)  that  was  developed  in  response  to  the  need  for  an  im¬ 
proved  display  of  altitude  and  vertical  speed  information  with  respect  to  system  goals 
to  support  V/STOL  aircraft  or  spacecraft  pilots  during  a  precision  vertical  descent 
task.  Explanations  of  the  proposed  integrated  flight  instrument  display  functionality, 
appearance,  and  usage  are  outlined,  and  the  rationale  behind  the  VAVI’s  design  is 
discussed. 


4.1  Overview  of  VAVI 

The  VAVI  was  developed  to  address  the  challenges  and  design  requirements  that  were 
revealed  through  the  cognitive  task  analysis  discussed  in  Chapter  3.  The  purpose  of 
the  VAVI  is  to  help  minimize  the  intermediary  cognitive  processes  that  are  currently 
required  to  get  from  perception  to  comprehension  and  projection  through  the  levels  of 
situational  awareness  by  directly  conveying  altitude  and  vertical  velocity  information 
to  indicate  unsafe  situations  and  hover-maneuvers  in  an  integrated  form  with  obvious 
cues.  It  is  intended  to  be  used  during  hover  and  vertical  descent  operations. 

The  VAVI  seeks  to  provide  an  intuitive  integrated  flight  instrument  display  that 
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leverages  ecological  perception  and  application  of  the  emergent  features  and  proximity 
compatibility  principle.  The  VAVI  consists  of  four  major  parts:  altitude  scale,  vertical 
speed  indicator,  vertical  speed  needle  and  symmetric  counterpart,  and  a  clock.  It  is 
intended  to  be  used  in  conjunction  with  other  flight  instrument  displays.  Figure  4-1 
illustrates  the  details  of  the  VAVI  in  context  of  whole  display. 


Figure  4-1:  Vertical  Altitude  and  Velocity  Indicator  (VAVI):  Visit  halab.mit.edu  for 
videos  of  the  VAVI 


4.2  VAVI  Integrated  Flight  Instrument  Elements 

This  section  describes  the  various  elements  of  the  VAVI.  Their  mapping  to  the  design 
requirements  in  Chapter  3  is  indicated  where  appropriate. 

4.2.1  Altitude  Scale 

The  center  vertical  shaft  is  a  fixed  altitude  bar  in  units  of  feet.  The  range  of  the 
altitude  is  dependent  on  the  altitude  at  which  an  air  or  spacecraft  would  initiate  a 
vertical  descent  or  hover.  The  VAVI  shown  here  illustrates  an  example  for  application 
of  hover  and  descent  operations  from  400  feet.  This  altitude  was  chosen  for  illustrative 
purposes.  Altitude  is  both  physically  and  conceptually  analog  in  terms  of  quantity 
and  representation.  Therefore,  the  analog  representation  of  altitude  in  the  VAVI  is 
most  appropriate  for  conveying  relative  height  information.  Likewise  the  direction 
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of  increasing  altitude  on  the  altitude  bar  is  consistent  with  the  direction  of  physical 
movement. 

The  small  gray  box  seen  here  at  97  feet,  outlines  the  current  altitude  as  a  digital 
value  that  will  slide  up  and  down  the  analog  altitude  scale  accordingly.  This  feature 
provides  the  altitude  to  a  high  precision  while  maintaining  the  analog  representation 
of  altitude.  In  this  version  of  the  VAVI,  when  the  altitude  exceeds  400  feet,  the  gray 
box  will  remain  at  the  top  of  the  fixed  bar,  while  the  digital  value  inside  the  box  will 
continue  to  increase  consistent  with  the  current  altitude  above  the  ground. 

4.2.2  Vertical  Speed  Indicator  (VSI) 

The  half  circle  to  the  right  of  the  altitude  bar  is  the  vertical  velocity  gauge  in  units 
of  feet  per  minute  (fpm)  and  depicts  vertical  speeds  between  ±500  fpm.  The  “arms” 
of  the  VAVI  refer  to  the  vertical  speed  needle  (the  right  arm)  and  its  symmetric 
counterpart  (the  left  arm).  The  VSI  moves  as  a  unit  vertically  along  the  altitude  bar 
such  that  the  arms  always  protrude  from  the  current  altitude  and  the  arm  to  the  left 
of  the  altitude  bar  is  symmetric  with  the  right  arm. 

In  conjunction  with  the  altitude  bar,  the  vertical  speed  indicator  needle  of  the 
graphical  display  becomes  a  visually  prominent  element  of  the  display  that  maps 
directly  to  the  system  states  that  are  important  to  the  operator.  The  position  of  the 
needle  provides  a  cue  as  to  what  state  the  vehicle  is  in,  including  hover  maneuvers 
and  precision  descents.  Therefore  the  arms  are  cues  that  provide  state  information 
at  a  glance.  A  nominal  ascent,  descent,  and  a  hover  maneuver  are  illustrated  in 
Figure  4-2a,  b,  and  c  respectively.  Instead  of  mentally  combining  the  altitude  and 
vertical  speed  in  an  effortful  approach  to  determine  the  vehicle  state,  the  operation 
can  now  be  carried  out  more  automatically.  The  VSI  in  conjunction  with  the  altitude 
bar  to  create  the  VAVI  directly  addresses  design  requirements  2  and  3  by  providing 
a  natural  depiction  of  vehicle  hover  state  and  utilizing  the  radar  altitude  in  close 
proximity  to  the  vertical  speed  in  order  to  combine  the  two  most  important  pieces  of 
goal-relevant  information  about  the  vehicle  state. 

A  key  feature  of  the  vertical  speed  indicator  element  of  the  VAVI  is  the  inclusion 
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of  unsafe  sink  rate  (negative  vertical  speed)  zones.  The  red  section  of  the  gauge 
indicates  an  unsafe  sink  rate  for  the  current  altitude  and  vehicle  and  environmental 
conditions.  When  the  vertical  speed  limit  is  violated  the  red  zone  becomes  a  brighter 
red,  but  could  be  indicated  in  a  number  for  implementation  in  current  displays.  This 
design  feature  directly  addresses  design  requirement  1  and  is  illustrated  in  Figure  4- 
2d.  By  including  the  dynamic  unsafe  zones,  the  pilot  no  longer  needs  to  recall  rules 
from  memory  to  determine  the  current  vehicle  limits  (see  Chapter  3).  A  simple  and 
commonly  taught  heuristic  was  employed  to  determine  a  safe  vertical  speed  and  thus 
the  location  of  the  red  zones:  the  vehicle  cannot  sink  faster  than  its  current  altitude. 
For  example,  if  the  vehicle  is  at  an  altitude  of  300  feet,  it  must  sink  no  faster  than 
300  fpm  in  order  to  safely  descend.  This  heuristic  holds  true  for  all  altitudes  greater 
than  or  equal  to  100  feet  at  which  point  100  fpm  remains  the  minimum  sink  rate.  A 
higher  fidelity  heuristic  may  be  employed  in  future  versions  of  the  VAVI  if  necessary 
however  this  simple  yet  reasonable  heuristic  demonstrates  the  concept  of  visualizing 
safety  constraints. 

4.2.3  Time  to  Touchdown  Clock 

The  box  below  the  altitude  bar  is  the  estimated  time  to  touchdown  as  a  function  of  the 
current  altitude  and  current  sink  rate.  This  box  only  appears  when  the  vertical  speed 
is  negative,  indicating  a  descent.  For  time  or  fuel  sensitive  operations,  this  piece  of 
information  can  be  critical.  In  the  past  this  calculation  has  been  a  rough  estimate  by 
pilots  or  astronauts  based  on  the  integration  of  altitude  and  vertical  speed  information 
over  time.  By  providing  this  information  directly  over  time,  mental  workload  can  be 
reduced  by  using  lower  levels  of  cognitive  control  to  discern  goal-relevant  information. 
This  element  of  the  VAVI  satisfies  design  requirement  4. 


4.3  Sensor  Requirements 

This  integrated  flight  instrument  is  designed  for  an  aircraft  or  spacecraft  equipped 
with  a  radar  altimeter  and  GPS  sensors  to  provide  accurate  altitude  and  positional 
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(c)  Hover  (d)  Unsafe  Descent 


Figure  4-2:  Vertical  Altitude  and  Velocity  Indicator 
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hover  accuracy.  The  VAVI  will  be  most  effective  when  implemented  with  an  in¬ 
stantaneous  vertical  speed  indicator  (IVSI)  which  displays  vertical  speed  without  the 
time  lag  normally  experienced  with  use  of  a  conventional  vertical  speed  indicator. 
Accelerometers  incorporated  into  the  linkage  allow  for  more  instantaneous  represen¬ 
tation  of  vertical  velocity  which  is  critical  for  precision  vertical  landing  operations 
and  ensuring  pilot  trust  in  the  data  display  [36] . 


4.4  Design  Principles 

Numerous  studies  have  shown  that  performance  can  improve  when  displays  utilize  di¬ 
rect  perception-action  visual  representations  that  allow  the  user  to  employ  the  more 
efficient  processes  of  perception  rather  than  the  cognitively  demanding  processes  in¬ 
volved  when  relying  on  memory,  integration,  and  inference  (e.g.  [37,  38,  39]).  Several 
design  principles  that  capture  this  concept  were  applied  to  the  design  of  the  VAVI. 
These  key  principles  are  provided  below  as  rationale  for  the  design  of  the  VAVI  and 
are  used  to  address  the  design  requirements  outlined  in  section  3.3. 

4.4.1  Ecological  Perception 

Known  for  his  research  in  theories  of  perception,  psychologist  James  J.  Gibson  es¬ 
tablished  the  concept  of  ecological  perception.  According  to  Gibson,  perception  is 
a  direct,  non- inferential  process.  In  other  words,  organisms  are  able  to  directly  per¬ 
ceive  the  environment  and  what  the  various  elements  of  the  environment  afford  them, 
without  making  associations  or  mediating  information  processing  [40].  Gibson  also 
states  that  perception  lies  on  a  continuum  from  direct  to  indirect  and  as  the  percep¬ 
tion  becomes  more  indirect,  the  information  available  for  perception  is  less  rich  and 
will  therefore  require  more  inference  [40] .  Because  direct  perception  does  not  require 
inference,  it: 

•  is  fast,  effortless,  and  proceeds  in  parallel  unlike  analytical  cognition  which  is 
slow  and  error-prone  [41], 
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•  can  be  substituted  for  higher,  more  demanding  cognitive  tasks  [39],  and 

•  produces  proficient  performance  from  lower  levels  of  cognitive  control  [41]. 

The  immense  power  of  people’s  perceptual  abilities  make  utilizing  direct  percep¬ 
tion  an  important  principle  in  the  design  of  user  displays  [41].  Direct  perception- 
action  interaction  compels  users,  in  this  case  pilots,  to  develop  realistic  system  men¬ 
tal  models  and  understand  design  constraints,  so  that  they  can  directly  perceive 
the  system  state  and  make  correct  decisions  [42].  The  integrated  movement  of  the 
current  altitude  display  and  the  vertical  speed  indicator  provide  a  direct  display  of 
both  pieces  of  information  in  a  meaningful  way,  thus  allowing  for  direct  perception  of 
critical  vehicle  states  and  corresponding  actions  neccessary  to  achieve  desired  states. 

4.4.2  Emergent  Features 

One  way  to  exploit  the  benefits  of  direct  perception  is  to  include  emergent  features  in  a 
display  design.  Bennett,  Toms,  and  Woods  defined  emergent  features  as  “high-level, 
global  perceptual  features  that  are  produced  by  the  interaction  among  individual 
parts  or  graphical  elements  of  a  display”  [43,  p.73].  While  each  graphical  element 
of  the  display  maintains  its  perceptual  identity,  the  emergent  features  evolve  from 
the  interaction  between  the  multiple  elements  [44,  37,  45].  Emergent  features  have 
been  shown  to  lead  to  improved  performance  (e.g.  [43,  38,  39]).  The  advantages  of 
emergent  features  are  that  they: 

•  highlight  critical  relationships  and  goal- relevant  system  states  [43,  38], 

•  map  to  system  constraints  [37,  43,  41], 

•  convey  information  about  processes  that  change  over  time  [38] , 

•  allow  the  user  to  quickly  and  with  low  cognitive  demand  discern  the  current 
state  of  the  system  [38] ,  and 

•  are  highly  salient  [37,  38]  such  that  the  “perception  of  the  individual  elements 
is  secondary  to  the  perception  of  the  object  itself”  [43,  p.74]. 
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The  vertical  speed  indicator  needle  and  its  symmetric  counterpart  form  the  key 
emergent  features  of  the  VAVI.  The  VAVI  arms  are  highly  salient  features  that  provide 
critical  vehicle  state  information  with  a  glance  and  correspond  to  specific  goal-relevant 
states  such  as  hovering  or  descending  at  a  specific  rate.  Likewise,  the  arms  also 
indicate  whether  or  not  vertical  speed  constraints  are  being  violated  by  their  position 
relative  to  the  unsafe  zone.  Thus  the  emergent  features  also  map  directly  to  system 
vertical  speed  constraints.  The  largest  benefit  of  the  VAVI  emergent  features  is  what 
it  affords  the  pilot  during  a  vertical  descent. 

4.4.3  Affordances 

The  concept  of  affordances  facilitates  the  connection  between  ecological  or  direct 
perception  and  the  design  of  user  displays.  As  previously  described,  humans  directly 
perceive  their  environment,  but  the  real  value  comes  from  perceiving  the  affordances 
of  the  environment.  Affordances  are  the  invariant  attributes  of  the  environment  that 
are  relevant  to  the  entity’s  purposes  and  therefore  specify  the  possibility  for  action  or 
suggest  appropriate  behavior  [46].  The  affordances  relate  perception  to  appropriate 
action,  eliminating  the  need  for  humans  to  infer  the  relationship  between  the  two. 

Affordances  are  demonstrated  by  the  VAVI  arms,  which  are  also  emergent  features. 
The  highly  salient  arms  afford  the  pilot  a  clear  understanding  of  vehicle  state  and 
specify  action  to  reach  the  goal  state  or  correct  for  an  unsafe  sink  rate. 

4.4.4  Proximity  Compatibility  Principle 

The  proximity  compatibility  principle  (PCP),  which  states  that  to  the  extent  that 
information  sources  must  be  integrated,  there  will  be  a  benefit  to  presenting  those 
sources  either  close  together,  in  an  objectlike  format,  or  by  configuring  them  to  create 
emergent  features  [47],  directly  facilitates  the  creation  of  emergent  features  [44, 48, 49], 
Two  aspects  of  proximity  explain  the  PCP.  The  first  dimension,  perceptual  proximity, 
refers  to  the  physical  closeness  of  two  display  components  that  convey  task-related 
information.  The  second  dimension,  processing  proximity,  is  the  extent  to  which 
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two  or  more  information  sources  are  used  as  part  of  the  same  task.  The  level  of 
processing  proximity  should  drive  the  level  of  display  proximity.  Therefore,  by  moving 
components  close  together,  the  designer  creates  psychological  closeness  or  perceptual 
similarity  [49]. 

PCP  has  proven  to  lead  to  better  performance  when  applied  to  information  sources 
that  require  integration  to  complete  a  task.  The  cognitive  benefits  of  employing  PCP 
in  display  design  include  the  following  [49]: 

•  the  visual  search  cost  is  decreased, 

•  the  information  access  cost  or  internal  movement  of  attention  is  decreased, 

•  computations  required  by  the  integration  of  information  decreases  the  load  on 
working  memory,  and 

•  the  need  to  retain  information  for  one  source  while  the  other  source  is  accessed 
is  eliminated. 

The  emergent  features  described  in  section  4.4.2  resulted  from  application  of  the 
proximity  compatibility  principle  in  which  proximity  was  accomplished  through  ob¬ 
ject  integration  and  increased  physical  proximity  of  the  two  sources  of  information: 
altitude  and  vertical  speed.  Specifically,  combining  the  VAVI  arms  with  the  vertical 
speed  indicator  exemplifies  application  of  the  proximity  compatibility  principle. 

4.4.5  Other  Design  Principles 

Several  other  design  principles  were  also  considered  during  the  design  of  the  VAVI 
and  are  described  here  briefly. 

The  dynamic  representation  of  quantitative  analog  data  such  as  altitude  and  ver¬ 
tical  speed  requires  the  designer  to  consider  the  user’s  physical  and  conceptual  repre¬ 
sentation  of  that  data.  According  to  Wickens  and  Hollands  [47] ,  altitude  information 
is  both  physically  and  conceptually  analog  and  therefore  the  altimeter  should  also 
be  an  analog  representation.  Digital  formats  provide  the  advantage  of  absolute  high 
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precision  values,  but  analog  representations  allow  for  more  direct  perception  of  rates 
of  change  and  relative  distance  from  a  limit  of  interest  [47], 

The  principle  of  the  moving  part  refers  to  making  the  motion  on  a  display  rep¬ 
resentative  of  the  motion  of  the  system  itself  [47].  In  other  words,  “the  direction  of 
movement  of  an  indicator  on  a  display  should  be  compatible  with  the  direction  of 
physical  movement”  [47,  p.135]  and  with  what  is  familiar  to  the  operator.  This  can 
refer  to  clockwise  versus  counterclockwise  motion,  up  and  down  motion,  or  others. 

Both  of  these  design  principles  were  exploited  in  the  design  of  the  VAVI  by  rep¬ 
resenting  both  altitude  and  vertical  speed  in  an  analog  form,  and  ensuring  that  the 
movement  of  those  values  is  consistent  with  the  normal  up/down  movement  of  altitude 
and  the  positive/negative  movement  of  vertical  speed. 


4.5  Cognitive  Processing  Model  Augmented  with 
the  VAVI 

The  cognitive  model  for  managing  altitude  and  vertical  speed  for  a  vertical  landing 
outlined  in  Chapter  3  illustrated  the  mental  steps  required  for  pilots  and  astronauts 
to  comprehend  the  perception  of  raw  data  from  the  outside  world,  displays,  and  their 
vestibular  systems  with  respect  to  safety  and  mission  goals  using  the  instruments 
currently  employed.  As  discussed  in  Chapter  3,  the  cognitive  workload  associated 
with  completing  those  steps  still  includes  numerous  cognitive  processes.  In  contrast, 
the  introduction  and  implementation  of  the  VAVI  in  air  and  spacecraft  vertical  dis¬ 
plays  allows  for  a  direct  perception-action  sequence  by  taking  advantage  of  ecological 
perception  through  emergent  features  created  by  the  object  integration  of  altitude 
and  vertical  speed  information. 

With  use  of  the  VAVI,  pilots  and  astronauts  can  directly  perceive  the  vehicle  state 
relative  to  the  safety  constraints  and  instantaneous  goals,  the  time  to  touchdown  as 
a  function  of  instantaneous  altitude  and  vertical  speed,  altitude  above  the  ground, 
and  the  vehicle  state  relative  to  the  goal  state.  Thus  level  1  SA  is  improved  by 
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effectively  eliminating  the  mentally-taxing  and  often  error-prone  tasks  associated  with 
pulling  from  long-term  and  working  memory  to  get  to  levels  2  and  3.  These  steps 
are  highlighted  in  Figure  4-3  which  compares  the  model  with  and  without  the  VAVI 
and  indicates  those  elements  of  the  cognitive  model  presented  in  Chapter  3  that  will 
be  replaced,  removed,  or  rearranged  as  a  result  of  use  of  the  VAVI.  These  changes 
are  described  below.  An  enlarged  view  of  the  modified  model  augmented  with  the 
VAVI  is  illustrated  in  Figure  4-4  and  indicates  the  direct  perception  of  goal-relevant 
information  in  level  1  SA  through  inputs  provided  by  the  VAVI. 

•  Replaced :  With  use  of  the  VAVI,  the  need  for  raw  data  indicators  such  as  a 
separate  altimeter  and  vertical  speed  indicator  is  eliminated  and  replaced  by 
the  VAVI. 

•  Removed-.  First,  the  pilot’s  understanding  of  current  vehicle  state  based  on 
currently  employed  displays  requires  the  mental  integration  of  a  digital-only  or 
analog  and  digital  combination  reprasentations  of  altitude  and  vertical  speed. 
The  VAVI,  however,  provides  salient  cues  to  relative  (analog)  and  exact  (digital) 
positional  information,  making  it  easy  to  quickly  glance  and  comprehend  the 
vehicle  state  relative  to  the  goal  state.  Therefore  this  step  from  perception  to 
comprehension  is  eliminated  with  use  of  the  VAVI  as  is  the  perception  of  the 
altimeter  value  and  vertical  speed  indicator  value. 

Second,  the  direct  display  of  unsafe  sink  rate  constraints  eliminates  the  need 
for  a  mental  heuristic  taught  or  learned  through  training  which  occupies  an  un¬ 
necessary  human  channel  of  information-processing.  Consequently  this  mental 
step  is  removed  from  the  mental  processes. 

Third,  by  directly  displaying  altitude  above  the  ground,  the  most  important 
altitude  during  a  vertical  descent  is  salient  and  clearly  displayed  in  an  integrated 
fashion  with  other  key  parameters.  Therefore  pilots  do  not  need  to  integrate 
barometric  and  ground  altitude  to  determine  altitude  above  the  ground  and  this 
step  is  removed  from  the  mental  processes. 
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Finally,  by  allowing  the  automation  to  do  the  integration  of  altitude  and  vertical 
speed  to  estimate  time  to  touchdown,  this  cognitive  step  is  eliminated. 

•  Rearranged :  By  eliminating  the  above  mental  steps,  the  VAVI  enables  direct 
perception  of  key  information.  For  example,  the  VAVI  enables  perception  of 
the  vehicle  state  relative  to  the  safety  constraints,  thus  moving  it  from  level  2 
situation  awareness  to  level  1.  Similarly,  the  vehicle  state  relative  to  a  goal  state 
such  as  hovering  is  perceived  as  is  the  time  to  touchdown  as  a  function  of  altitude 
and  vertical  speed,  and  altitude  above  the  ground.  While  terrain  texture  and 
shadow  size  will  still  be  perceived  and  later  comprehended  to  provide  altitude 
information,  that  same  information  can  be  directly  perceived  by  way  of  the 
VAVI  in  conjunction  with  the  outside  cues,  or  in  their  absence. 

As  illustrated  in  Figure  4-4,  the  VAVI  does  not  address  every  cognitive  process 
associated  with  the  transition  from  perception  to  comprehension  during  a  vertical 
descent.  Hovering  and  vertical  landings  are  still  challenging  tasks  for  pilots  of  heli¬ 
copters,  V/STOL  aircraft,  and  spacecraft,  but  the  VAVI  is  designed  to  reduce  the 
mental  workload  that  corresponds  to  the  cognitive  processes  outlined  above.  In  doing 
so,  it  creates  a  mental  reserve  that  can  be  applied  to  more  critical  tasks  that  displays 
and  automation  cannot  directly  address. 
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Chapter  5 


Human  Performance 
Experimentation 


Human  performance  experimentation  was  conducted  using  a  commercial  flight  sim¬ 
ulation  software  package  (Microsoft  Flight  Simulator  2004)  modified  to  display  the 
VAVI  in  a  simulated  V/STOL  aircraft  head-up  display  (HUD).  The  experiment  tested 
hover  and  landing  performance  hypotheses  using  a  HUD  including  the  VAVI,  and  a 
conventional  HUD  without  the  VAVI.  The  experimental  method,  including  partici¬ 
pants,  apparatus,  experimental  tasks,  and  experimental  design  are  described  in  this 
chapter. 


5.1  Experiment  Objectives 


The  objectives  of  this  experiment  focus  on  the  use  of  the  VAVI  for  precision  vertical 
operations  by  rotorcraft,  V/STOL  aircraft,  and  spacecraft.  The  specific  objectives 
are  to  test  the  effectiveness  of  the  VAVI  for  improving  hover  and  landing  performance, 
and  decreasing  mental  workload  during  these  operations. 
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5.2  Experimental  Hypotheses 


5.2.1  Hover  Performance 

The  ability  to  maintain  a  constant  hover  altitude  was  hypothesized  to  be  most  influ¬ 
enced  by  the  emergent  features,  namely  the  arms  of  the  VAVI,  that  indicate  hover 
state  tied  directly  to  the  hover  altitude.  A  hover  situation  is  clearly  illustrated  by 
outstretched  and  level  VAVI  arms.  Likewise,  capturing  an  accurate  hover  altitude 
(e.g.  stop  at  a  pre-specified  altitude)  is  supported  by  the  VAVI  arms  which  are  flat 
when  a  hover  has  been  achieved.  To  help  achieve  that  hover,  the  relative  position 
of  the  arms  takes  advantage  of  ecological  perception  and  allows  the  pilot  to  more 
quickly  perceive  and  anticipate  rates  to  determine  when  to  terminate  vertical  speed 
to  reach  the  desired  altitude.  The  following  hypotheses  capture  the  expected  hover 
performance: 

•  Hypothesis  1 :  the  ability  to  maintain  a  constant  hover  altitude  is  expected  to 
improve  with  use  of  the  VAVI  as  compared  to  use  of  conventional  hover  and 
landing  flight  displays 

•  Hypothesis  2:  use  of  the  VAVI  is  expected  to  improve  the  task  of  capturing  an 
accurate  hover  altitude 

5.2.2  Landing  Performance 

Vertical  landing  performance  is  a  function  of  two  parameters:  the  ability  to  maintain 
an  accurate  descent  rate  and  to  maintain  a  safe  descent  rate.  As  the  VAVI  directly 
indicates  safe  descent  rate  limits  and  clearly  alerts  the  pilots  of  violations  of  those 
constraints  through  a  change  in  color,  it  was  expected  that  the  VAVI  would  enable 
more  direct  perception  of  the  vehicle  state  in  relation  to  the  constraints.  Likewise,  the 
arms  of  the  VAVI  which  indicate  both  a  relative  vertical  speed  direction  and  rate  on  an 
analog  scale,  clearly  display  exact  values  making  it  simple  to  realize  the  appropriate 
direction  of  control  input  for  correction.  The  following  hypotheses  capture  expected 
landing  performance: 
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•  Hypothesis  3:  the  ability  to  maintain  a  safe,  dynamic  sink  rate  is  hypothesized 
to  improve  with  use  of  the  VAVI  over  conventional  flight  display  components 

•  Hypothesis  4‘  use  of  the  VAVI  is  expected  to  result  in  fewer  errors  when  at¬ 
tempting  to  control  a  constant  static  vertical  speed 

5.2.3  Workload 

As  discussed  in  Chapter  3,  the  mental  steps  required  to  get  from  the  data  that  is 
provided  on  the  existing  landing  displays  to  the  information  needed  by  pilots  to 
determine  if  the  vehicle  is  conforming  to  safety  constraints  is  a  mentally  demanding 
task.  Since  the  VAVI  allows  for  direct  perception  of  this  information  and  a  display  of 
the  safety  constraints,  some  of  the  mental  steps  required  to  comprehend  important 
attributes  of  the  system  are  eliminated  (see  Chapter  4).  Therefore  the  following 
results  are  expected: 

•  Hypothesis  5:  a  reduction  in  mental  workload  for  all  hover  and  vertical  landing 
tasks  is  expected  with  use  of  the  VAVI  as  compared  to  conventional  displays 


5.3  Participants 

A  total  of  31  participants  participated  in  this  experiment,  28  men  and  3  women.  Of 
those  31  participants,  9  (8  men  and  1  female)  did  not  meet  the  training  proficiency 
levels  outlined  in  section  5.7  and  therefore  did  not  move  on  to  the  test  scenarios. 
Participants  were  recruited  based  on  their  rotor- wing  or  V/STOL  pilot  experience, 
or  PC-based  flight  simulator  experience.  The  participant  population  included  stu¬ 
dents,  recreational  pilots,  and  professional  pilots.  Participants  were  compensated  for 
their  participation  with  an  MIT  Humans  and  Automation  Laboratory  t-shirt  and 
refreshments. 

The  age  range  of  qualifying  participants  was  12-63  years  with  a  average  age  of  34. 
Eight  of  the  participants  had  both  fixed-wing  and  rotor-wing  pilot  experience,  while 
3  had  rotor-wing  and  3  had  fixed-wing  only  experience.  A  total  of  8  only  had  PC 
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flight  simulator  experience.  Of  the  14  that  had  actual  flight  experience,  9  of  them 
also  had  PC  flight  simulator  experience.  The  number  of  hours  of  experience  in  each 
participant’s  most  experienced  platform  (rotor-wing,  fixed-wing,  or  PC  simulator) 
ranged  from  15-9000  with  the  median  being  300  hours  and  a  mean  of  1175  hours. 
Appendix  F  outlines  more  details  of  the  study  demographics. 


5.4  Test  Bed 

5.4.1  Apparatus 

The  human  participant  experimentation  of  the  VAVI  was  conducted  using  a  Dell  19” 
flat  panel  monitor  operated  at  1152  x  864  pixels  and  a  16-bit  color  resolution.  The 
workstation  was  a  Dell  OptiPlex  GX520  with  an  Intel  Pentium  4  3.40GHz  proces¬ 
sor  and  an  Intel  83945G  Express  Chipset  Family  graphics  card.  System  audio  was 
provided  using  standard  workstation  speakers. 

Participants  interacted  with  the  simulator  via  a  Saitek  X45  digital  joystick  and 
throttle  control,  and  CH  Products  CH  Pro  rudder  pedals,  both  shown  in  Figure  5-1. 
The  entire  experiment  apparatus  was  portable  and  designed  to  be  positioned  at  any 
table  or  desk  with  a  height  adjustable  chair.  An  example  workstation  is  illustrated 
in  Figure  5-2. 

5.4.2  Simulation  Platform 

Microsoft  Flight  Simulator  2004  with  an  Abacus  Military  Aircraft  Collector’s  Edition 
expansion  pack  was  used  to  simulate  an  AV-8B  Harrier  V/STOL  jet  aircraft.  Details 
about  the  flight  simulator  settings  can  be  found  in  Appendix  A. 

Two  Harrier  HUDs  were  created  from  scratch:  one  with  the  VAVI  (Figure  5-3) 
and  one  without  the  VAVI  (Figure  5-4).  Display  elements  were  modeled  after  an 
actual  Harrier  head-up  display.  The  VAVI  display  replaced  any  analog  or  digital 
representations  of  vertical  velocity  and  radar  altitude  used  in  the  HUD  without  the 
VAVI.  As  noted  in  Figure  5-3,  the  conventional  display  represents  altitude  digitally 
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(a)  Joystick  and  Throttle  (b)  Rudder  Pedals 

Figure  5-1:  Control  Devices 


and  vertical  speed  both  analog  and  digitally,  but  as  separate  components  of  the 
display. 


5.5  Experimental  Task 

5.5.1  Scenario  Commonalities 

Two  test  scenarios  were  designed  for  this  experiment.  Each  test  scenario  involved 
flying  an  AV-8B  Harrier  from  Green  State  Airport  in  Providence,  RI  to  Logan  Inter¬ 
national  Airport  in  Boston,  MA.  This  route  was  chosen  for  its  relatively  short  distance 
and  a  VHF  Omnidirectional  Range  (VOR)  navigation  system  which  made  finding  the 
airport  very  simple.  The  flight  between  Providence  and  Boston,  after  takeoff  and  be¬ 
fore  hover  and  landing,  lasted  approximately  four  minutes.  This  allowed  participants 
to  relax  after  takeoff  and  prepare  for  the  landing,  yet  not  expend  too  much  time  on 
the  straight  and  level  portion  of  flight  which  was  not  of  interest  in  this  experiment. 
The  target  landing  site  at  Logan  International  Airport  was  a  helipad  100  m  in  diam¬ 
eter  located  between  the  two  main  runways  and  marked  by  a  white  circle  with  a  large 
“H”  in  the  middle.  The  center  of  the  helipad  was  the  target  landing  site. 
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Figure  5-2:  Apparatus 
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Figure  5-3:  Harrier  Head-Up  Display  with  VAVI  Display 
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Figure  5-4:  Harrier  Conventional  Head-Up  Display 
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Every  scenario  was  conducted  at  dusk  in  a  simulated  summer  environment.  This 
operating  condition  was  chosen  because:  1)  the  dark  blue  sky  at  dusk  provided  better 
contrast  with  the  white  HUD  components,  making  it 'clearer  for  the  participants,  and 
2)  the  diminishment  of  visual  cues  at  this  time  of  day  in  conjunction  with  the  already 
reduced  visual  cues  provided  by  a  simulated  synthetic  outside  view,  replicated  an 
environment  in  which  instruments  are  more  critical  for  hovering  and  landing  tasks. 

In  both  scenarios,  participants  were  asked  to  maintain  a  specific  heading  in  order 
to  get  to  the  Boston  airport.  Participants  were  also  asked  to  stay  within  1000-2000 
feet  altitude  above  the  ground  and  less  than  450  knots  of  airspeed  during  the  straight 
and  level  portion  of  flight  to  ensure  they  were  not  too  high  or  too  fast  upon  approach 
to  the  airport  for  a  vertical  landing.  The  takeoff  and  straight  and  level  flight  were 
not  of  importance  in  either  scenario. 

5.5.2  Scenario  Variations 

The  two  test  scenarios  varied  slightly  during  the  vertical  descent  phase.  The  pri¬ 
mary  differences  were  the  altitude  at  which  the  participants  were  asked  to  hover,  and 
the  descent  rate  they  were  asked  to  maintain.  Both  scenarios  included  a  stationary 
hover  (at  different  altitudes),  while  one  scenario  asked  the  participants  to  maintain 
a  constant  static  descent  rate  of  -100  feet-per-minute  (fpm)  to  the  landing  site,  the 
other  required  a  dynamic  sink  rate,  which  was  always  less  than  the  vehicle’s  current 
altitude  (e.g.  if  the  vehicle  is  at  150  feet,  it  should  not  sink  any  faster  than  150  feet). 
This  variation  is  referred  to  as  the  vertical  speed  heuristic.  The  two  test  scenarios 
in  Figure  5-5  illustrate  the  differences  between  the  two  test  scenarios.  The  shaded 
circles  included  on  the  descent  indicate  a  hover,  while  the  text  boxes  to  the  right  of 
the  circles  indicate  the  altitude  at  which  the  hover  should  occur,  and  the  length  of 
time  the  hover  should  be  maintained.  Participants  were  told  to  hover  at  the  assigned 
hover  altitude,  but  that  if  they  were  unable  to  stop  at  that  altitude  on  the  descent, 
they  should  stabilize  out  as  soon  as  they  could  and  begin  their  hover  at  that  altitude 
rather  than  try  to  climb  back  up  to  the  assigned  hover  altitude.  They  were  also 
told  to  indicate  to  the  experimenter  when  they  wanted  to  begin  their  hover  and  the 
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experimenter  informed  the  participant  when  20  seconds  had  passed. 

Table  5.1  outlines  the  flight  tasks  during  the  two  test  scenarios  and  highlights 
the  differences  between  them.  Participants  were  not  given  a  priority  for  these  task 
objectives. 


HDG  =  041 
1000-2000  ft  AGL 


HOG  =  041 
1000-2000  ft  AGL 


Figure  5-5:  Experimental  Test  Scenarios 


Table  5.1:  Flight  Tasks 


Flight  Task 

Objectives 

Test  1 

Test  2 

Conventional  Takeoff 

None 

X 

X 

Cruising 

None 

X 

X 

Hover 

Maintain  precise  commanded  altitude 

X 

X 

Capture  accurate  commanded  altitude 

X 

X 

Vertical  Landing 

Maintain  accurate  static  descent  rate 
Maintain  accurate  dynamic  descent  rate 

X 

X 

Maintain  precise  vertical  speed  control 

X 

X 

5.6  Experimental  Design 

This  experiment  was  a  2(Display)  x  2(Flight  Task)  mixed  design  study  within-subjects 
on  the  flight  task  factor,  and  between-subjects  on  the  display  factor. 

5.6.1  Independent  Variables 

Two  independent  variables  were  of  interest  in  this  experiment:  instrument  display 
component  and  flight  task.  Instrument  display  component  refers  to  the  HUD  that 
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participants  used  throughout  the  experiment.  Participants  either  saw  the  VAVI  HUD 
illustrated  in  Figure  5-3  or  the  conventional  HUD  illustrated  in  Figure  5-4.  Therefore, 
this  two  level  factor  was  a  between-subjects  variable.  The  flight  task  factor  included 
hovering  and  landing,  was  a  within-subjects  variable  such  that  every  participant  was 
asked  to  perform  both  hovers  and  landings. 

5.6.2  Dependent  Variables 

Several  dependent  variables  were  used  in  this  experiment  to  measure  hover  and  land¬ 
ing  performance  and  workload.  Each  of  those  variables  is  described  in  detail  in  this 
section. 

Hover  Accuracy 

Hover  accuracy  addresses  the  participant’s  ability  to  capture  a  commanded  hover 
altitude,  and  addresses  hypothesis  2.  Stopping  and  hovering  at  the  desired  hover  alti¬ 
tude  is  an  important  skill  that  requires  the  use  of  accurate  altitude  and  vertical  speed 
information.  Hover  accuracy  is  measured  as  the  difference  between  the  commanded 
hover  altitude  and  the  actual  hover  altitude.  The  actual  hover  altitude,  defined  as 
the  altitude  that  the  participant  tried  to  maintain,  was  determined  to  be  the  altitude 
at  the  time  that  the  participants  indicated  they  were  beginning  their  hover.  For  this 
measure,  a  smaller  value  corresponds  to  better  performance. 

Hover  Precision 

Hover  precision  addresses  hypothesis  1  which  corresponds  to  the  ability  to  maintain 
a  precise  hover.  Deviations  from  the  actual  hover  altitude  are  captured  using  a  root 
mean  square  error.  The  root  mean  square  error,  shown  in  Equation  5.1,  is  a  common 
measure  of  success  that  gives  the  error  value  the  same  dimensionality  as  the  actual 
and  desired  values  [50].  The  smaller  the  RMSE,  the  better  the  performance.  In 
the  measure  of  hover  precision,  the  “desired”  variable  corresponds  to  the  altitude  at 
the  first  second  of  the  self-initiated  20  second  hover  and  “actual”  corresponds  to  the 
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altitude  at  every  second  during  that  20  second  interval. 


RMSE  = 


Y  ( actual  —  desired )2 


n 


(5.1) 


Vertical  Speed  Precision 

Vertical  speed  precision  refers  to  the  precision  control  of  the  vertical  speed  during 
the  descent.  This  measure  addresses  the  vertical  descent  hypotheses  3  and  4  and 
refers  to  the  ability  to  maintain  both  a  dynamic  and  static  descent  rate.  The  ability 
to  maintain  a  static  descent  rate  was  also  measured  using  a  RMSE,  as  described 
in  Equation  5.1.  The  RMSE  of  vertical  speed  from  the  completion  of  the  hover  to 
landing  was  calculated.  Figure  5-6  illustrates  how  the  static  descent  rate  was  analyzed 
using  an  example  vertical  speed  profile.  The  solid  straight  line  at  -100  fpm  indicates 
the  desired  vertical  speed  and  the  dotted  lines  on  either  side  of  that  line  represent 
±  10%  of  the  desired  vertical  speed.  FAA  practical  test  standards  do  not  explicitly 
outline  a  rate  of  closure  or  vertical  speed  control  standard,  so  ±  10%  was  determined 
to  be  a  reasonable  range  based  on  the  results  of  this  experiment.  Participants  were 
not  penalized  for  being  within  10%  of  the  commanded  static  descent  rate  (Figure  5- 
6A),  and  participants  were  also  not  penalized  for  having  a  positive  (>  0  fpm)  vertical 
speed  (Figure  5-6B)  indicating  that  they  were  climbing  and  not  descending.  To 
prevent  penalizing  the  participants  for  having  a  positive  vertical  speed,  those  data 
log  entries  corresponding  to  a  positive  vertical  speed  were  not  considered  in  the  RMSE 
calculation.  Finally,  those  vertical  speeds  that  were  outside  of  the  10%  range  were 
penalized  only  for  their  difference  from  the  ±  10%  range  around  the  commanded 
speed  (Figure  5-6C). 

The  measure  of  a  participant’s  ability  to  maintain  a  dynamic  sink  rate  used  a  sim¬ 
ilar  approach  as  depicted  in  Figure  5-7.  In  this  case,  participants  were  not  penalized 
for  being  within  ±  10  feet  per  minute  of  the  current  commanded  vertical  speed  (Fig¬ 
ure  5-7A),  which  is  equivalent  to  the  negative  current  altitude.  Again,  any  vertical 
speed  outside  of  this  range  was  penalized  only  for  its  deviation  from  the  outer  limit 
of  the  range  (Figure  5-7C).  Like  the  measure  of  static  descent  rate,  participants  were 
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Figure  5-6:  Static  Vertical  Speed  Analysis 

not  penalized  for  having  a  positive  vertical  speed  (Figure  5-7B)  and  those  values  were 
not  included  in  the  RMSE  calculation. 

Descent  Duration  Error 

The  descent  duration  is  a  measure  of  the  comparison  between  the  time  that  it  should 
have  taken  the  participant  to  descend,  had  they  descended  according  to  the  com¬ 
manded  dynamic  or  static  vertical  speed  heuristic,  versus  the  actual  time  of  descent. 
This  dependent  variable  is  another  measure  of  landing  performance  which  captures 
the  participants’  overall  ability  to  maintain  a  specified  descent  rate.  A  smaller  value 
for  this  measure  indicates  better  adherence  to  consistent  accurate  vertical  speed  con¬ 
trol. 

Workload  Measures 

To  measure  subjective  workload,  participants  rated  their  perceived  mental  workload 
on  a  ten-point  scale,  with  1  corresponding  to  minimal  to  no  mental  workload,  and  10 
corresponding  to  the  highest  mental  workload  the  participant  has  ever  experienced. 
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Time  (sec) 


Figure  5-7:  Dynamic  Vertical  Speed  Analysis 

A  ten-point  scale  was  chosen  to  mimic  the  ten-point  mental  demand  scale  associated 
with  the  NASA  TLX  subjective  workload  measure. 


5.7  Procedure 

Participants  were  first  welcomed  by  the  experimenter  and  given  a  brief  introduction 
to  the  experiment.  Participants  then  completed  a  signed  consent  form  (Appendix  B), 
followed  by  a  demographic  survey  which  gathered  information  regarding  previous 
flight  or  flight  simulator  experience  (Appendix  C).  Participants  were  then  given  a 
tutorial  on  a  set  of  PowerPoint  slides  that  detailed  the  purpose  of  the  experiment  and 
described  the  Harrier  and  the  specifics  of  the  joystick  and  rudder  pedals.  Tutorials 
(Appendix  D)  were  created  for  each  of  the  two  types  of  flight  instrument  displays 
(VAVI  or  conventional)  and  each  participant  saw  only  the  tutorial  relevant  to  the 
HUD  they  would  utilize  throughout  the  experiment.  The  experiment  tutorials  took 
approximately  10-15  minutes  to  complete. 

Participants  were  then  given  a  series  of  training  lessons  to  learn  how  to  fly  the 
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simulated  Harrier.  These  lessons  involved  first  viewing  a  conceptual  flight  task  lesson 
on  the  tutorial  slides,  then  practicing  the  flight  task  using  the  simulator  experimental 
apparatus.  All  participants  had  up  to  50  minutes  to  demonstrate  their  proficiency 
at  each  of  the  specified  tasks  (hovering  and  vertical  landings).  Table  5.2  outlines 
the  proficiency  level  required  for  each  task  in  order  to  move  on  to  the  test  scenarios. 
Generally  participants  took  90%  of  the  entire  training  time  to  reach  the  proficiency 
levels. 


Table  5.2:  Training  Proficiency  Measures 

Task 

Proficiency 

Hovering 

Maintain  a  commanded  hover  altitude  within  +/-  10  ft  for  30 
seconds 

Vertical  Landing 

Land  at  the  Boston  airport 

Perform  a  controlled  descent 

Do  not  crash 

Following  the  training  and  demonstration  of  proficiency,  participants  completed 
the  two  test  scenarios  described  in  the  previous  section,  each  lasting  approximately 
10  minutes.  Before  beginning  the  tests,  participants  were  informed  that  their  perfor¬ 
mance  during  takeoff  and  straight  and  level  portions  of  flight  would  not  be  measured, 
but  hovering  and  vertical  landing  performance  would  be  measured  relative  to  the  ob¬ 
jectives  outlined  in  Table  5.1.  During  each  test  scenario,  the  figure  accompanying  it 
(see  Figure  5-5)  was  taped  at  the  bottom  of  display  as  a  reminder  of  the  specific  goals 
for  each  flight.  Participants  were  presented  the  test  scenarios  in  randomized  order  to 
prevent  a  possible  order  effect.  At  the  completion  of  each  test  scenario,  participants 
filled  out  a  ten-point  workload  survey.  Once  the  experiment  was  completed,  feedback 
was  solicited  about  the  display  and  the  experience  through  a  semi-structured  inter¬ 
view  to  help  guide  the  discussion  (see  Appendix  E  for  initial  interview  questions). 
The  two  test  scenarios  followed  by  the  workload  scales  took  a  total  of  30  minutes  on 
average.  The  entire  experiment  took  roughly  90  minutes  total. 
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5.8  Data  Collection 


During  testing,  key  flight  parameters  such  as  altitude,  heading,  pitch,  bank,  vertical 
velocity,  airspeed,  latitude  and  longitude,  and  others  were  recorded  to  a  data  log 
file  every  second  using  flight  data  recorder  software.  The  experimenter  also  observed 
each  test  participant  during  their  test  scenarios  and  noted  any  interesting  behavior 
to  include  gaze  or  focus,  body  posture,  and  comments  made  throughout  or  during  a 
specific  task. 
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Chapter  6 


Results 


This  chapter  presents  the  statistical  results  of  the  experiment  described  in  Chapter  5. 
The  experiment  included  two  independent  variables:  Head-Up  Display  Instrument 
(VAVI  or  conventional)  and  Flight  Task  (hovering  or  landing).  Numerous  dependent 
variables  were  considered  in  the  analysis  of  the  data  in  order  to  capture  and  measure 
hover  performance,  landing  performance,  and  workload  as  described  in  Chapter  5. 
Each  of  those  variables  are  discussed  and  presented  in  this  chapter. 


6.1  Overview 

The  dependent  measures  used  to  analyze  the  two  flight  tasks  were  often  different  for 
each  task  because  the  dependent  measures  used  to  measure  hover  performance  were 
not  extendable  to  landing  performance  and  vice  versa.  Therefore,  the  general  linear 
model  used  for  this  analysis  included  both  single  and  two-factor  analysis  of  variance 
as  applicable.  Appendix  G  gives  a  summary  with  more  details  of  all  of  the  statistical 
tests  used  in  the  analysis  of  these  results.  The  independent  factors  were  considered 
to  be  fixed  and  the  participants  were  a  random  factor.  For  all  reported  results,  a  = 
0.05  unless  stated  otherwise. 
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6.2  Hover  Performance 


6.2.1  Hover  Accuracy 

As  outlined  in  Section  5.6.2,  the  measure  for  determining  the  participants’  ability  to 
capture  a  commanded  hover  altitude  was  hover  altitude  error,  or  the  error  between 
actual  hover  altitude  and  commanded  hover  altitude.  One-way  analysis  of  variance 
(AN OVA)  results  indicated  that  there  was  no  significant  difference  in  hover  accuracy 
between  flight  display  levels  (F(l,34)  =  0.425,  p  =  0.519).  A  square  root  transfor¬ 
mation  of  the  data  with  a  0.5  correction  due  to  zero  values  was  required  to  meet 
homogeneity  and  normality  assumptions  for  the  AN OVA  test.  The  boxplot  in  Fig¬ 
ure  6-1  shows  the  median  hover  altitude  error  as  well  as  the  quartiles  and  extreme 
values  for  each  flight  display  level,  and  Table  6.1  summarizes  the  key  statistics. 


Figure  6-1:  Hover  altitude  error 


Table  6.1:  Hover  Altitude  Error  Summary 


Mean 

Median 

Std  Dev 

VAVI 

17.16 

16.43 

No  VAVI 

20.59 

IEd 

18.35 

90 


6.2.2  Hover  Precision 


A  one-way  ANOVA  performed  on  the  root  mean  square  error  (RMSE)  of  the  hover 
altitude  during  the  20  seconds  that  the  participants  informed  the  experimenter  that 
they  were  hovering  indicated  that  there  was  also  no  significant  difference  in  hover 
precision  between  flight  display  levels  (F(l,34)  =  1.484,  p  =  0.231).  As  there  were 
normality  violations,  a  square  root  transformation  of  the  data  was  required  to  meet 
homogeneity  and  normality  assumptions.  Figure  6-21  illustrates  the  comparison  be¬ 
tween  the  two  flight  display  levels  and  Table  6.2  outlines  the  key  statistics. 


Figure  6-2:  RMSE  of  hover  altitude 


Table  6.2:  RMSE  of  hover  altitude  summary 


Mean 

Median 

Std  Dev 

VAVI 

3.73 

2.57 

3.29 

No  VAVI 

5.11 

4.80 

3.41 

1  “18”  is  a  statistical  outlier  and  refers  to  the  spreadsheet  line,  not  the  dependent  measure  value 
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6.3 


Landing  Performance 


6.3.1  Vertical  Speed  Precision 

The  RMSE  of  vertical  descent  speed  was  analyzed  as  a  2x2  ANOVA,  treating  flight 
display  (VAVI  or  No  VAVI)  as  the  primary  factor,  and  vertical  speed  heuristic  (dy¬ 
namic  or  static)  as  the  secondary  factor.  Vertical  speed  heuristic  refers  to  whether 
participants  were  asked  to  hold  a  constant  (static)  vertical  speed  or  a  changing  (dy¬ 
namic)  vertical  speed  during  their  descent.  A  reciprocal  transformation  of  the  data 
was  necessary  to  meet  homogeneity  and  normality  requirements.  The  RMSE  of  ver¬ 
tical  speed  was  significantly  lower  with  the  VAVI  (F(l,30)  =  5.484,  p  =  0.026),  but 
there  was  no  significant  difference  between  vertical  speed  heuristics  (F(l,30)  =  0.023, 
p  =  0.879).  There  was  no  significant  interaction  between  the  factors.  Figure  6-3  illus¬ 
trates  the  comparison  between  the  flight  display  levels  and  test  scenarios.  Table  6.3 
outlines  the  respective  means,  medians,  and  standard  deviations  for  each  test  with 
and  without  the  VAVI. 


13 
u 

?  100  00-1 


p  -  0.026  VAVI  vs.  Mo  VAVI 
p  *  •  J79  Dynamic  v«.  Static 


I  1 
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Vertical  Speed  Heuristic 


HUD 
■  vavi 

0  No  VAVI 


Figure  6-3:  RMSE  of  vertical  descent 
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Table  6.3:  RMSE  of  vertical  speed  summary 


VAVI 

No  VAVI 

Dynamic 

Static 

Dynamic 

Static 

Mean 

62.50 

64.78 

122.99 

111.39 

Median 

49.07 

45.54 

81.73 

99.79 

Std  Dev 

41.69 

40.77 

91.14 

80.07 

6.3.2  Descent  Duration  Error 

The  descent  duration  error  was  defined  as  the  difference  between  the  desired  descent 
duration  and  the  actual  descent  duration,  normalized  by  the  total  duration.  It  mea¬ 
sures  the  “smoothness”  of  the  descent.  Non-parametric  tests  were  used  to  determine 
differences  between  flight  display  level  and  vertical  speed  heuristic.  A  Mann- Whitney 
U  test  was  conducted  that  compared  flight  display  levels.  There  was  no  difference  in 
this  landing  performance  measure  between  VAVI  and  non-VAVI  users  (p  =  0.790).  A 
Wilcoxon  Signed-Rank  test  was  performed  on  the  data  to  compare  the  vertical  speed 
heuristic  that  participants  were  asked  to  use.  Results  show  that  there  is  a  highly 
significant  difference  between  heuristics  (p  =  0.001),  indicating  that  performance  was 
much  better  when  participants  were  asked  to  maintain  a  constant  static  vertical  speed 
on  their  descent.  This  difference  is  illustrated  in  Figure  6-4  and  outlined  in  Table  6.4. 


Dynamic  S»dk 

Vertical  Speed  Heuristic 


Figure  6-4:  Descent  Duration  Error 
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Table  6.4:  Descent  Duration  Error  Summary 


VAVI 

No  VAVI 

Dynamic 

Static 

Dynamic 

Static 

Mean 

4.70 

0.80 

6.96 

0.71 

Median 

5.12 

0.91 

6.18 

0.70 

Std  Dev 

2.64 

0.30 

4.76 

0.19 

6.4  Workload 


Subjective  workload  was  measured  through  use  of  a  ten-point  workload  scale  that 
was  modeled  after  the  NASA  TLX  mental  demand  measure.  Participants  completed 
a  workload  survey  following  each  test  scenario. 


6.4.1  10-Point  Workload  Scale 

The  10-pt  anchored  workload  scale  associated  a  “10”  with  the  highest  mental  workload 
that  a  person  has  ever  experienced,  and  a  “1”  with  minimal  to  no  mental  workload. 
A  non-parametric  Mann- Whitney  U  test  was  conducted  across  all  test  scenarios  to 
compare  flight  display  levels.  The  U  test  indicated  marginal  significance  (p  =  0.108)2, 
but  the  boxplots  in  Figure  6-5  illustrate  that  the  median  values  for  the  10-pt  workload 
rating  is  higher  without  the  VAVI  in  both  test  scenarios  (see  Table  6.5). 

A  Wilcoxon  Signed-Rank  test  was  used  to  analyze  the  differences  between  test 
scenarios  and  showed  no  significance  between  vertical  speed  control  heuristics  (p  = 
0.642). 


Table  6.5:  10-pt  Subjective  Workload  Rating  Summary 


VAVI  No  VAVI 


Test  1 

Test  2 

Test  1 

Test  2 

Mean 

5.13 

5.00 

5.70 

5.70 

Median 

5.00 

4.50 

7.00 

6.50 

Std  Dev 

1.51 

1.35 

2.50 

2.41 

2a=0.1  for  non-parametric  tests 
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Figure  6-5:  10-pt  Workload  Scale  Boxplot 


6.5  Top  Performer  Subset  Analysis 


A  further  analysis  of  all  the  dependent  variables  was  conducted  on  the  top  three 
performers  from  each  flight  display  group  (VAVI  and  No  VAVI).  The  top  performers 
were  determined  based  on  their  average  ranking  across  all  dependent  measures  as 
illustrated  in  Appendix  H.  This  analysis  was  conducted  to  investigate  the  performance 
between  participants  who  set  themselves  apart  from  the  rest  of  the  participant  pool 
by  demonstrating  controlled  and  expert  flight  performance.  This  subset  analysis  may 
more  closely  represent  the  use  of  these  displays  in  operational  aircraft,  since  these 
participants  were  able  to  control  the  aircraft  in  a  simulated  environment  similar  to 
what  would  be  expected  from  highly  trained  pilots.  Using  only  those  six  participants 
who  set  themselves  apart  from  the  majority  of  the  participants,  the  same  analyses 
were  conducted  on  all  dependent  measures  discussed  above.  Non-parametric  analyses 
were  used  for  all  dependent  measures  of  this  participant  subset. 

In  this  subset  of  expert  participants,  hover  accuracy  continued  to  show  no  signifi¬ 
cant  difference  between  flight  displays  (Mann- Whitney,  p  =  0.589).  Hover  precision, 
however,  showed  a  marginally  significant  difference  in  flight  displays  (Mann- Whitney, 
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p  =  0.1263),  as  compared  to  the  non-significant  difference  previously  outlined  for  all 
participants  (p  =  0.231).  As  illustrated  in  Figure  6-6b,  expert  users  with  the  VAVI 
showed  improved  ability  to  maintain  a  desired  altitude  as  compared  to  the  conven¬ 
tional  display  of  vertical  speed  and  altitude.  Table  6.6  summarizes  the  key  statistics 
for  hover  performance  of  this  participant  subset. 

10.00-  __ 


8.00  4 


VAVI  No  VAVI  VAVI  No  VAVI 


HUD  HUD 

(a)  Hover  Accuracy  (b)  Hover  Precision 

Figure  6-6:  Top  Participants  Hover  Performance  Measures 


Table  6.6:  Top  Participants  Statistics  Summary,  Hover  Measures 


VAVI 

No  VAVI 

Dep.  Measure 

Mean 

Median 

Std  Dev 

Mean 

Median  Std  Dev 

Hover  Accuracy 

8.17 

4.00 

11.50 

Hover  Precision 

1.81 

1.21 

1.42 

5.23  3.87 

As  indicated  in  Table  6.9,  vertical  speed  precision  was  still  highly  significant 
(Mann-Whitney,  p  =  0.015)  between  flight  display  levels  in  this  subset,  and  still 
not  significant  between  vertical  speed  heuristics  (Wilcoxon,  p  =  0.600).  The  plot  of 
means  illustrated  in  Figure  6- 7b  demonstrates  the  consistent  improvement  in  perfor¬ 
mance  across  both  vertical  speed  control  tasks.  The  consistent  VAVI  results  between 
vertical  speed  heuristics  indicate  that  while  the  control  of  a  dynamic  vertical  speed 
3<y—0. 1  for  non-parametric  tests 
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RMSE  of  vertical  speed  during  the  descent 
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Figure  6-7:  Top  Participants  Landing  Performance  Measures 
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is  more  difficult  than  that  of  a  static  vertical  speed,  the  VAVI  makes  both  tasks 
consistently  easier  (as  illustrated  by  better  performance  and  significantly  less  vari¬ 
ation)  as  shown  in  Figure  6-7a.  Results  for  descent  duration  error  were  consistent 
with  the  findings  for  all  participants:  there  was  no  significance  between  flight  display 
levels  (Mann- Whitney,  p  =  0.699),  and  a  significant  difference  between  vertical  speed 
heuristics  (Wilcoxon,  p  =  0.028).  Tables  6.7  and  6.8  summarize  the  mean,  median, 
and  standard  deviation  of  all  of  the  landing  performance  dependent  measures  of  this 
participant  subset. 


Table  6.7:  Top  Participants  Statistics  Summary,  Dynamic  Vertical  Speed  Heuristic 


Dep.  Measure 

VAVI 

No  VAVI 

Mean 

Median 

Std  Dev 

Mean 

Median 

Std  Dev 

VS  Precision 

30.50 

26.58 

8.89 

68.26 

57.45 

28.75 

Descent  Duration  Error 

4.63 

5.06 

2.66 

5.90 

6.18 

2.00 

10-pt  Workload 

6.00 

7.00 

1.73 

7.33 

7.00 

0.58 

Table  6.8:  Top  Subjects  Statistics  Summary,  Static  Vertical  Speed  Heuristic 


Dep.  Measure 

VAVI 

No  VAVI 

Mean 

Median 

Std  Dev 

Mean 

Median 

Std  Dev 

VS  Precision 

35.24 

40.42 

12.34 

105.10 

54.36 

101.60 

Descent  Duration  Error 

0.91 

0.91 

0.15 

0.66 

0.69 

0.06 

10-pt  Workload 

5.33 

6.00 

1.15 

7.33 

7.00 

0.58 

Workload  results  show  a  statistically  significant  difference  between  flight  displays 
in  this  expert  subset  (Mann- Whitney,  p  =  0.026)  as  compared  to  a  marginally  sig¬ 
nificant  difference  in  the  large  participant  pool  (p  =  0.108),  and  still  no  significance 
between  vertical  speed  heuristics  (Wilcoxon,  p  =  0.157).  The  boxplot  in  Figure  6- 
8  illustrates  the  difference  in  workload  between  flight  displays.  This  indicates  that 
when  participants  have  adequate  training  and  are  comfortable  and  good  at  the  ver¬ 
tical  landing  tasks,  workload  is  reduced  with  use  of  the  VAVI.  Interestingly,  while 
the  quantitative  results  indicate  that  a  dynamic  vertical  speed  is  more  difficult  to 
control  than  a  static  speed,  participants  did  not  perceive  a  significant  difference  in 
their  mental  workload  between  vertical  speed  conditions. 


98 


•4X3  -  J-  -1- 

- 1 - r~ 

Dynamic  Static 

Vertical  Speed  Heuristic 


HUD 

VAVI 

B  Mo  VAVI 


Figure  6-8:  Top  Participants  Workload  Performance  Measures 


6.5.1  Top  Performer  Summary 

To  summarize  the  performance  of  the  top  six  subjects,  Table  6.9  outlines  the  statistical 
significance  associated  with  each  dependent  measure.  More  detailed  non-parametric 
results  are  outlined  in  Appendix  F.  Hover  accuracy,  vertical  speed  precision,  and 
descent  duration  error  results  were  consistent  with  results  from  the  large  participant 
group.  Hover  precision,  however,  showed  marginally  significant  improvement  with 
use  of  the  VAVI,  while  workload  results  strongly  indicated  a  lower  mental  workload 
associated  with  the  VAVI.  These  results  indicate  improved  hover  performance  and 
reduced  perceived  workload  with  use  of  the  VAVI  when  used  by  an  expert  subset  of 
participants. 


Table  6.9:  Top  Participants  Summary 


Dep.  Measure 

HUD 

VS  Heuristic 

Hover  Accuracy 

p  =  0.589 

N/A 

Hover  Precision 

p  =  0.126 

N/A 

VS  Precision 

p  =  0.015 

p  =  0.600 

Descent  Duration  Error 

p  =  0.699 

p  =  0.028 

10-pt  Workload 

p  =  0.026 

p  =  0.157 
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Chapter  7 


Discussion 


This  chapter  discusses  the  results  presented  in  Chapter  6  and  compares  them  to  the 
hypotheses  outlined  in  Chapter  5.  First,  hover  performance,  landing  performance, 
and  workload  results  will  be  discussed  in  relation  to  the  model  presented  in  Chap¬ 
ter  3.  Then,  subjective  responses  gathered  through  a  semi-structured  interview  will 
be  reported  and  discussed. 

7.1  Hover  Performance 

Hover  performance  was  classified  by  hover  accuracy  and  hover  precision.  As  out¬ 
lined  in  section  5.6.2,  hover  accuracy  addresses  the  ability  to  capture  a  commanded 
hover  altitude  during  a  descent,  and  hover  precision  captures  the  ability  to  main¬ 
tain  a  constant  hover  altitude.  Though  the  results  did  not  indicate  any  statistically 
significant  differences  between  flight  display  factor  levels  at  the  a  =  0.05  level,  they 
are  still  consistent  with  hypotheses  1  and  2.  As  outlined  in  Tables  6.1  and  6.2,  the 
mean  and  median  hover  altitude  error  values  for  those  participants  that  used  the 
VAVI  was  considerably  less  than  for  those  who  did  not,  signifying  a  possible  trend 
towards  significance.  In  the  analysis  of  the  top  six  performers,  hover  precision  showed 
marginal  significance  in  better  performance  with  use  of  the  VAVI,  further  illustrating 
the  effectiveness  of  the  VAVI. 

Hover  performance  was  a  difficult  metric  to  capture  because  the  20  seconds  during 
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which  the  participants  indicated  to  the  experimenter  that  they  were  hovering,  did  not 
necessarily  capture  the  participants’  best  hover  performance  throughout  the  cycle. 
This  difficulty  may  have  had  a  significant  affect  on  the  results  of  this  dependent 
variable. 


7.2  Landing  Performance 

Landing  performance  showed  clearer  results,  particularly  that  the  VAVI  corresponds 
to  significantly  less  vertical  speed  deviations  for  both  static  and  dynamic  sink  rate 
control  over  the  duration  of  the  descent.  These  results  are  consistent  with  hypotheses 
3  and  4  and  demonstrate  that  average  vertical  speed  control  error  over  the  duration 
of  the  descent  is  less  with  use  of  the  VAVI  than  without.  The  direct  perception  of 
both  relative  and  more  precise  vertical  speed  facilitated  by  the  VAVI  enabled  more 
consistent  monitoring  of  this  parameter  and  provided  a  more  obvious  display  of  large 
or  undesirable  deviations  in  vertical  speed.  By  displaying  flight  parameters  relative  to 
the  goal  state,  the  vehicle  state  was  directly  perceived  and  comprehended  in  a  single 
step  as  outlined  in  cognitive  model  in  Chapter  4. 

The  descent  duration  error  landing  performance  measure  illustrated  a  very  signifi¬ 
cant  difference  between  vertical  speed  heuristics  despite  the  lack  of  difference  between 
flight  displays  at  the  a=0.05  level.  This  exemplifies  that  the  consistent  control  of  a 
static  vertical  descent  is  much  easier  for  participants  than  the  control  of  a  constantly 
changing  desired  vertical  speed,  regardless  of  the  display  that  is  available.  This  is 
an  expected  result  because  a  constant  rate  of  descent  is  easier  to  remember  than  a 
constantly  changing  one  that  requires  adjustment.  This  measure  can  be  thought  of 
as  a  measure  of  the  “smoothness”  of  the  descent.  Further  analysis  of  the  data  re¬ 
vealed  that  the  descent  duration  error  for  dynamic  vertical  speed  control  was  most 
often  associated  with  a  shorter  duration,  meaning  that  participants  touched  down 
too  quickly.  This  is  also  a  logical  and  expected  result. 

The  analysis  of  this  variable  using  the  top  six  performers  illustrated  a  much  more 
pronounced  (though  not  statistically  significant)  difference  between  flight  display  lev- 
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els  in  the  dynamic  vertical  speed  heuristic  scenario,  which  is  more  consistent  with 
expected  results.  Without  a  display  of  safe  vertical  speed  constraints  on  the  conven¬ 
tional  display,  participants  were  less  aware  of  their  excessive  vertical  speed,  which 
resulted  in  too  fast  of  a  descent.  In  addition,  the  rapidly  changing  nature  of  variables 
that  must  constantly  be  tracked  in  the  dynamic  situation  requires  more  iterations 
through  the  levels  of  situation  awareness  outlined  in  Chapter  3.  By  enabling  the  di¬ 
rect  perception  and  comprehension  of  key  dynamic  parameters,  several  mental  steps 
were  removed  with  use  of  the  VAVI  which  allowed  for  more  appropriate  aircraft  con¬ 
trol  more  quickly. 


7.3  Workload 

A  subjective  workload  measure  was  used  to  capture  the  mental  workload  that  partic¬ 
ipants  associated  with  their  tasks.  Subjective  workload  measures  can  provide  some 
information  about  workload,  but  are  also  difficult  to  employ  because  different  people 
judge  the  meaning  of  the  scale  differently. 

The  results  of  the  ten-point  workload  scale,  which  showed  that  the  VAVI  marginally 
corresponds  to  a  lower  mental  workload  than  a  conventional  display,  are  consistent 
with  hypothesis  5.  The  lack  of  difference  in  results  between  test  scenarios  indicates 
that  participants  perceived  differences  in  hover  altitude  and  vertical  speed  heuris¬ 
tic  to  be  equivalently  mentally  demanding.  However,  across  all  flight  tasks,  those 
participants  who  used  the  VAVI  did  consistently  report  experiencing  a  lower  mental 
workload.  Since  this  measure  was  a  between-subjects  measure,  the  results  suggest  a 
decrease  in  perceived  mental  workload  between  flight  display  levels.  The  objective 
measures  of  hover  and  landing  performance  suggest  that  participants  had  to  perform 
significantly  less  highly  cognitive  calculations  and  could  rely  more  on  perception  with 
use  of  the  VAVI.  This  allowed  for  more  precise  control  and  these  subjective  results 
also  support  that  conclusion.  For  the  expert  participants,  however,  the  difference  in 
perceived  mental  workload  between  flight  displays  was  statistically  significant.  Over¬ 
all,  the  results  indicate  a  lower  mental  workload  associated  with  the  VAVI,  which  is 
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more  obvious  among  expert  participants. 


7.4  Summary 

Although  not  all  dependent  measures  of  hover  and  landing  performance  and  men¬ 
tal  workload  resulted  in  statistical  significance  at  the  a=0.05  level,  all  of  the  results 
illustrate  a  trend  towards  improved  performance  with  use  of  the  VAVI  as  seen  in 
summary  of  boxplots  in  Figure  7-1.  Furthermore,  Figure  7-2  depicts  the  comparison 
of  means,  medians,  and  standard  deviations  between  flight  display  levels  for  all  de¬ 
pendent  measures  and  test  scenarios.  The  similar  line  trends  which  illustrate  better 
performance  with  the  VAVI  across  all  dependent  variables,  provide  another  indication 
of  the  consistent  and  positive  results  with  the  VAVI. 
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Figure  7-1:  Performance  Measures  Summary 


7.5  Subjective  Responses 


General  feedback  was  elicited  from  the  participants  upon  completion  of  the  experi¬ 
ment.  The  specific  questions  that  were  used  for  discussion  are  outlined  in  Appendix  E. 
The  responses  pertaining  directly  to  the  display  of  vertical  speed  and  altitude,  as  well 
as  other  comments  about  the  experiment  and  display  as  a  whole  are  discussed  here. 

7.5.1  Usability  Survey 

Those  participants  who  used  the  VAVI  indicated  that  they  found  several  character¬ 
istics  very  helpful.  Specifically,  many  of  the  participants  reported  that  the  red  zone 
“jumped  out”  at  them  when  it  changed  color  and  indicated  a  necessary  increase  in 
throttle.  In  this  way,  the  VAVI  indicated  not  only  the  system  constraints,  but  also 
mapped  to  a  specific  control  input  to  fix  the  vertical  speed  violation.  This  observation 
supports  the  model  in  Chapter  4  which  illustrates  the  reduced  mental  steps  necessary 
to  quickly  transition  to  an  appropriate  action  as  a  result  of  good  situation  awareness 
with  use  of  the  VAVI.  Many  of  the  participants  also  indicated  that  the  VAVI  was 
specifically  helpful  for  trying  to  maintain  an  accurate  hover  altitude  by  trying  to  keep 
the  VAVI  arms  level.  When  asked  how  they  used  the  VAVI,  several  participants  said 
they  looked  at  the  direction  of  the  needle  to  get  a  relative  sense  of  vertical  speed. 
One  participant  specifically  stated  that  he  used  the  VAVI  to  keep  from  descending 
too  fast.  Another  stated  that  he  found  the  fact  that  the  arms  moved  up  and  down 
with  the  current  altitude  grabbed  his  attention  during  flight  and  made  him  aware  of 
his  current  altitude. 

Roughly  half  of  the  participants  indicated  that  they  only  used  the  right  arm, 
while  the  other  half  said  they  used  both  arms  of  the  VAVI  as  a  unit.  Of  those  that 
used  both,  several  indicated  that  they  used  it  in  the  periphery  of  their  vision.  Of 
those  that  did  not  use  the  left  arm,  two  indicated  that  they  did  not  even  notice  it, 
and  only  one  found  the  left  arm  distracting.  Almost  all  of  the  participants  reported 
that  the  VAVI  was  not  useful  during  the  straight  and  level  portion  of  flight  due  to 
the  quick  up  and  down  movement  of  the  vertical  speed  indicator  arms  with  changes 
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in  pitch.  The  VAVI  was  originally  only  intended  to  be  displayed  during  hover  and 
landing  maneuvers  and  not  en-route  flight,  but  simulation  constraints  drove  the  need 
to  display  it  all  the  time  in  these  experiments.  Moreover,  many  participants  reported 
that  when  the  arms  were  pegged  up  or  down  (i.e.  at  >  500  fpm  or  <  -500  fpm)  it 
was  difficult  to  tell  which  direction  they  were  pointed  with  a  glance  and  thus  were 
not  referenced  as  frequently  when  they  were  in  this  position,  even  if  this  occurred 
during  the  vertical  portion  of  flight.  Similarly,  one  participant  pointed  out  that  when 
vertical  speed  with  less  than  100  or  -100  fpm,  it  was  difficult  to  tell  an  exact  vertical 
speed.  Current  displays,  however,  do  not  provide  vertical  speed  information  to  any 
more  precision  than  the  VAVI. 

Participants  in  the  conventional  display  group  also  discussed  the  processes  that 
they  went  through  to  understand  vertical  speed  and  position  information.  Three 
participants  stated  that  they  only  used  the  digital  readout  of  vertical  speed,  while 
only  one  preferred  to  only  use  the  analog.  The  majority  looked  at  the  analog  scale  first 
and  then  to  the  digital  value  for  more  precise  information  depending  on  the  position 
of  the  pointer  along  the  analog  scale.  One  participant  indicated  that  he  would  like  the 
digital  value  to  be  in  the  pointer  so  that  he  would  not  have  to  move  back  and  forth  to 
line  up  speed  with  altitude.  He  followed  this  comment  by  stating  that  “the  trick  is  to 
be  able  to  see  instantly,  the  position  of  things.”  Four  of  the  participants  who  used  the 
conventional  display  specifically  reported  that  they  found  the  tasks  difficult  because 
they  focused  on  one  variable  and  another  would  slip.  One  participant  indicated  that 
by  the  time  he  realized  that  his  vertical  speed  was  no  longer  under  control,  he  was 
already  on  the  ground.  He  suggested  changing  the  color  of  the  altimeter  box  when 
the  altitude  dropped  below  an  appropriate  pre-determined  altitude  to  indicate  that 
the  control  of  vertical  speed  is  important. 

Interestingly,  the  majority  of  comments  pertaining  to  possible  improvements  of  the 
conventional  display  were  directly  related  to  VAVI  characteristics  specifically  designed 
to  eliminate  the  cognitive  demand  associated  with  vertical  operations.  Since  all  of 
the  information  is  combined,  the  VAVI  prevents  the  need  to  focus  on  one  parameter 
to  the  detriment  of  another  as  many  of  the  participants  reported  while  using  the 
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conventional  display.  In  addition,  several  of  the  conventional  display  participants 
specifically  indicated  the  need  for  some  sort  of  alerting  method  which  the  VAVI 
directly  addresses  through  the  use  of  the  unsafe  sink  rate  zone. 

When  asked  what  other  information  they  wished  they  had  to  help  them  with  the 
vertical  descent  and  hovering,  the  participants  indicated  they  needed  longitudinal  and 
lateral  movement  and  position  in  relation  to  the  target  landing  site.  Many  participants 
wished  they  had  a  top-down  view  of  the  landing  site  in  order  to  see  below  them.  One 
participant  reported  that  he  would  like  to  have  an  altitude  bug  which  he  could  set  to 
indicate  the  altitude  at  which  he  would  like  to  hover  so  that  he  did  not  have  remember 
and  remind  himself  of  this  information.  This  feature  was  originally  included  in  the 
VAVI  design,  but  it  was  not  included  in  the  VAVI  experimentation  due  to  software 
implementation  constraints.  An  example  of  this  feature  will  be  depicted  in  the  next 
section.  Two  other  participants  also  indicated  that  they  would  have  liked  airspeed  to 
be  an  analog  display  as  well. 

7.6  Recommendations 

7.6.1  Design  Recommendations 

Based  on  comments  elicited  at  the  completion  of  the  experiment,  the  following  VAVI 
design  recommendations  should  be  considered  for  future  versions  of  the  VAVI. 

•  When  vertical  speed  exceeds  ±  500  fpm,  arms  should  provide  an  obvious  display 
of  the  vertical  speed  direction.  One  suggestion  is  to  use  the  left,  symmetric  arm 
to  indicate  position  through  a  change  in  color,  the  addition  of  an  arrowhead 
(Figure  7-3a),  dotting  of  the  line,  or  perhaps  flashing. 

•  To  provide  a  better  visualization  of  vertical  speed  indicator  needle  angle  and 
thus  more  precise  vertical  speed  information,  a  dotted  reference  line  at  0  fpm 
could  be  added  as  illustrated  in  Figure  7-3b. 

•  When  vertical  speed  exceeds  ±  500  fpm,  the  digital  readout  should  remain 
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consistent  with  the  analog  scale  discretization  of  vertical  speed.  For  example, 
instead  of  displaying  the  exact  digital  value  of  vertical  speed  once  it  exceeds  the 
analog  scale,  the  digital  value  should  only  increase  in  increments  of  100  fpm  as 
is  consistent  with  the  analog  scale. 

•  An  altitude  cue  which  can  be  set  by  the  user  could  be  included  as  an  optional 
feature  for  accurate  hovering.  The  cue  would  provide  a  reminder  of  the  tar¬ 
get  hover  altitude.  One  option  is  to  highlight  the  desired  hover  altitude  with 
a  dotted  box  around  it,  include  an  arrow-like  cue,  or  both,  as  illustrated  in 
Figure  7-3c. 

•  The  VAVI  should  be  used  in  conjunction  with  a  display  of  lateral  and  longitu¬ 
dinal  position  relative  to  the  target  landing  site.  This  display  would  required 
GPS  technology  to  map  positional  error.  This  concept  is  depicted  in  Figure  7- 
3d  where  the  center  of  the  cross  is  the  target  landing  site  and  the  small  circle 
is  the  aircraft  position  relative  to  the  target. 

•  To  further  integrate  important  landing  information,  a  heading  arc  could  be 
incorporated  over  the  top  of  the  VAVI. 

•  The  VAVI  could  include  a  display  of  acceleration  to  provide  additional  direc¬ 
tional  and  rate  information. 

7.6.2  Future  Experiment  Recommendations 

PC  flight  simulators  provide  an  excellent  starting  point  for  the  test  and  evaluation 
of  display  concepts.  However,  flight  simulation  cannot  replace  real  aviation.  In  this 
experiment,  numerous  participants  with  vertical  aircraft  flight  experience  were  unable 
to  gain  adequate  proficiency  with  the  PC  simulator,  which  they  generally  attributed  to 
the  lack  of  visual  cues  and  haptic  feedback.  Participants  with  a  large  amount  of  video 
game  experience  had  the  easiest  time  adjusting  to  the  lack  of  haptic  feedback  and 
small  field  of  view  as  a  result  of  their  familiarity  with  synthetic  gaming  environments. 
This  illustrates  the  importance  of  trying  to  recreate  a  realistic  environment  for  testing 
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Figure  7-3:  VAVI  Design  Recommendations 
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of  displays  with  domain  experts.  Non-PC  simulators  that  are  used  for  training  and 
logging  hours  provide  a  higher  level  of  realistic  visual  cues  and  flight  dynamics  which 
PC  simulators  cannot  provide.  Some  of  the  instabilities  observed  in  the  PC  simulator 
version  of  the  Harrier  appear  to  be  unique  to  the  Microsoft  PC  flight  simulator  used  in 
this  experiment.  Therefore,  future  experiments  to  test  the  effectiveness  of  the  VAVI 
should  focus  on  using  non-PC,  high-fidelity  simulators  such  as  those  that  are  used  to 
train  helicopter  and  V/STOL  pilots. 
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Chapter  8 


Conclusions 


There  is  an  increasing  requirement  for  precision  vertical  landing  capability  for  air  and 
spacecraft,  which  is  driven  by  the  advantages  of  vertical  takeoff  and  landing  capa¬ 
bility  and  improvements  in  sensor  and  control  technology.  Pilots  of  vertical  landing 
vehicles  face  numerous  control  challenges  which  often  involve  the  loss  of  outside  vi¬ 
sual  perceptual  cues  or  the  control  of  flight  parameters  within  tight  constraints.  The 
challenging  conditions  that  accompany  vertical  flight,  in  particular  the  hovering  and 
landing  aspects  of  vertical  flight,  place  high  cognitive  demands  on  pilots.  Current 
flight  instrument  displays,  which  are  important  for  mitigating  these  challenges,  still 
have  high  cognitive  demand  and  lack  perceptual  decision  aids  specifically  for  hovering 
and  landing  tasks.  Therefore  this  research  was  motivated  by  the  desire  to  reduce  the 
mental  workload  and  improve  performance  of  precision  vertical  operations  of  air  and 
spacecraft. 

The  design  and  testing  of  the  Vertical  Altitude  and  Velocity  Indicator  demon¬ 
strates  the  possibility  for  improved  mitigation  of  the  challenges  which  often  accom¬ 
pany  vertical  flight.  Most  importantly,  the  VAVI  may  improve  the  safety  of  vertical 
landings  and  hovering,  while  also  creating  cognitive  reserves  which  can  be  applied 
to  the  mission  rather  than  the  control  of  the  aircraft.  Likewise,  NASA  has  plans  to 
return  to  the  Moon  and  land  with  higher  precision  than  ever  before,  but  experimental 
results  show  that  this  may  not  be  possible  without  an  ecological  perceptual  aid  for 
the  astronauts. 
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8.1  Research  Objectives  and  Findings 


The  objectives  of  this  research  were  to  investigate  the  cognitive  processes  of  pilots 
and  astronauts  during  a  vertical  landing,  develop  a  vertical  precision  landing  aid,  and 
evaluate  the  effectiveness  of  the  landing  aid.  The  goal  was  to  address  these  objectives 
through  the  following  methods: 

•  Conduct  a  cognitive  task  analysis  and  develop  a  cognitive  model  for  vertical 
landing  and  hover  operations  to  establish  design  requirements  for  vertical  land¬ 
ing  instruments  (see  Chapter  3). 

•  Design  an  integrated  flight  instrument  display  component  that  addresses  the 
design  requirements  and  follows  proven  design  principles  (see  Chapter  4). 

•  Use  human-performance  experimentation  to  evaluate  the  effectiveness  of  the 
new  vertical  precision  landing  aid  on  hover  and  vertical  landing  performance 
(see  Chapters  5-7). 

The  cognitive  model  for  vertical  landing  and  hover  operations  in  Chapter  3  illus¬ 
trated  the  numerous  mentally  demanding  steps  required  to  proceed  through  the  levels 
of  situation  awareness  and  act  accordingly.  Chapter  4  depicted  the  reduction  in  cogni¬ 
tive  processes  associated  with  use  of  the  VAVI  by  enabling  direct  perception-action. 
The  proposed  cognitive  model  was  then  supported  through  the  human  participant 
experimentation  of  the  VAVI  which  indicated  improved  performance  and  a  reduction 
in  workload  as  a  result  of  the  removal  of  unnecessary  cognitive  processing  steps. 

The  VAVI  design  and  subsequent  human-performance  testing  resulted  in  some 
very  positive  findings  which  establish  that  the  VAVI  does  in  fact  improve  hover  and 
landing  performance  and  reduces  mental  workload.  The  statistically  significant  differ¬ 
ence  between  flight  displays  for  precise  vertical  speed  control  indicated  that  the  VAVI 
improves  both  dynamic  and  static  vertical  speed  control.  Other  marginally  statisti¬ 
cally  significant  landing  performance  metrics  illustrate  that  the  VAVI  aids  with  more 
“smooth”  control  of  vertical  speed  during  landing  by  enabling  direct-perception  ac¬ 
tion  of  the  current  vehicle  state  and  provides  more  consistent  situation  awareness 
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of  changing  vehicle  limits,  large  deviations  from  intended  states,  and  possible  con¬ 
straint  violations.  Hover  performance  metrics,  while  not  statistically  significant  at 
the  a=0.05  level,  also  illustrate  a  trend  of  better  performance  with  use  of  the  VAVI. 
Workload  results  indicate  a  marginally  significant  reduction  in  workload  with  use  of 
the  VAVI,  which  is  more  significant  among  expert  users. 

The  most  revealing  result  of  the  experiment,  despite  the  lack  of  statistical  signif¬ 
icance  at  the  a  =  0.5  level,  is  the  consistency  with  which  the  VAVI  corresponds  to 
better  performance  (as  a  function  of  mean,  median,  and  standard  deviation)  across 
all  test  scenarios  for  every  dependent  measure  (as  illustrated  in  Figure  7-2  in  Chap¬ 
ter  7).  In  addition,  the  results  for  the  expert  participants  who  most  closely  model  the 
pilots  expected  to  use  the  VAVI,  illustrate  a  consistent  and  effective  increase  in  pre¬ 
cise  landing  performance  across  all  tasks.  While  the  results  do  not  all  show  statistical 
significance,  a  positive  trend  would  likely  improve  with  additional  testing. 


8.2  Recommendations  and  Future  Work 

Though  the  results  of  this  thesis  indicate  that  the  VAVI  shows  potential  for  improved 
performance  and  reduced  mental  workload  for  precision  vertical  landings,  further 
investigation  is  warranted.  The  following  are  recommendations  for  future  follow-on 
work  based  on  the  research  presented  in  this  thesis. 

•  The  VAVI  was  designed  to  be  used  specifically  during  vertical  descent  or  hover 
operations.  Therefore,  the  VAVI  should  be  displayed  only  when  certain  vehicle 
requirements  are  met,  which  indicate  that  a  vertical  descent  or  hover  is  about 
to  be  executed.  The  VAVI  should  appear  to  the  pilot  when  it  is  most  appro¬ 
priate,  and  not  during  straight  and  level  flight.  Further  investigation  into  the 
appropriate  logic  for  introduction  of  the  VAVI  (fading  in  and  out,  etc.)  during 
flight  should  be  conducted. 

•  To  gain  a  more  detailed  understanding  of  how  the  VAVI  is  being  used  by  pilots, 
similar  testing  with  use  of  an  eye-tracking  system  could  be  performed.  Specifi- 
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cally  of  interest  is  the  application  of  the  left,  symmetric  VAVI  arm  and  relative 
versus  precise  parameter  monitoring. 

•  The  design  recommendations  outlined  in  section  7.6.1  that  resulted  from  sub¬ 
jective  participant  responses  should  be  further  investigated  and  addressed. 

•  Testing  of  the  VAVI  in  a  high-fidelity  V/STOL  or  helicopter  simulator  is  re¬ 
quired  to  determine  its  effectiveness  in  a  more  realistic  flight  setting. 

The  VAVI  should  be  integrated  into  an  already  existing  display  to  replace  vertical 
speed  and  radar  altitude  information.  The  displays  utilized  for  experimentation  in 
this  thesis  were  modeled  after  a  typical  V/STOL  head-up  display.  However,  specific 
integration  and  placement  of  the  VAVI  will  be  aircraft  dependent.  Specific  aircraft 
requirements  should  be  taken  into  consideration  in  any  future  design  and  testing  of 
the  VAVI. 
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Appendix  A 

Microsoft  Flight  Simulator  2004 
Settings 


This  appendix  outlines  the  control  sensitivities  and  realism  settings  of  Microsoft  Flight 
Simulator  2004  (MSFS)  that  were  applied  in  the  human  performance  experimentation 
of  the  VAVI.  These  settings  are  used  to  make  the  control  of  an  aircraft  in  MSFS  as 
realistic  as  possible. 

A.l  Control  Sensitivities 

The  control  sensitivity  settings  determine  the  feedback  of  the  flight  controls.  Fig¬ 
ure  A-l  illustrates  a  screenshot  of  the  control  device  sensitivity  settings.  The  pedal 
sensitivity  was  set  low  due  to  inherent  over-sensitivity  of  the  pedals.  The  joystick  was 
sensitivity  was  set  high  to  be  as  realistic  as  possible.  The  null  zones  for  both  devices 
were  zero  to  remove  any  dead  space  in  the  control  movements. 

A.  2  Realism 

MSFS  realism  settings  control  the  flight  model  used  in  the  simulation.  Figure  A-2 
illustrates  a  screenshot  of  the  realism  settings  used  for  this  flight  simulator  experiment. 
All  levels  of  realism  were  set  to  average  in  order  to  provide  a  challenging  yet  acceptable 


117 


(a)  Joystick  Sensitivity  (b)  Pedals  Sensitivity 


Figure  A-l:  Control  Sensitivities 
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Figure  A-2:  Microsoft  Flight  Simulator  Realism  Settings 


level  of  difficulty  for  the  participants.  These  realism  settings  primarily  addressed  the 
ease  of  takeoff  and  straight  and  level  flight  which  was  not  of  interest  in  this  experiment. 
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Appendix  B 


Consent  to  Participate 


The  following  consent  to  participate  was  signed  by  all  participants  prior  to  taking 
part  in  the  human  performance  experiment. 
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CONSENT  TO  PARTICIPATE  IN 
NON-BIOMEDICAL  RESEARCH 


An  Ecological  Perceptual  Aid  for  Lunar  and  Earth-Based  Vertical  Precision  Landing 

You  are  asked  to  participate  in  a  research  study  conducted  by  Cristin  Smith,  from  the  Aeronautics  and 
Astronautics  Department  at  the  Massachusetts  Institute  of  Technology  (M.I.T.)  The  results  of  this  study  will  be 
contributed  to  a  graduate  thesis.  You  were  selected  as  a  possible  participant  in  this  study  because  you  either 
have  flight  experience  or  flight  simulator  (PC-based)  experience.  You  should  read  the  information  below,  and 
ask  questions  about  anything  you  do  not  understand,  before  deciding  whether  or  not  to  participate. 

•  PARTICIPATION  AND  WITHDRAWAL 

Your  participation  in  this  study  is  completely  voluntary  and  you  are  free  to  choose  whether  to  be  in  it  or  not.  If 
you  choose  to  be  in  this  study,  you  may  subsequently  withdraw  from  it  at  any  time  without  penalty  or 
consequences  of  any  kind.  The  investigator  may  withdraw  you  from  this  research  if  circumstances  arise  which 
warrant  doing  so. 

.  PURPOSE  OF  THE  STUDY 

The  purpose  of  this  study  is  to  investigate  the  effectiveness  of  a  combined  vertical  speed  and  altitude  indicator 
for  hover  aircraft  vertical  descents  to  landing.  It  is  of  interest  to  determine  if  human  performance,  in  terms  of 
safe  and  effective  operations,  improves  with  the  use  of  this  display  element. 

.  PROCEDURES 

If  you  volunteer  to  participate  in  this  study,  we  would  ask  you  to  do  the  following  things: 

•  Participate  in  a  training  session  to  become  familiar  with  a  Microsoft  Flight  Simulator  Harrier  platform 
and  the  corresponding  control  devices  such  as  a  joystick  and  collective 

•  Practice  using  the  simulator  will  continue  until  you  demonstrate  an  ability  to  perform  certain  flight  tasks 
such  as  straight  and  level  flight,  hovering,  and  landing 

•  Execute  test  sessions 

•  Attend  a  debriefing  to  determine  your  subjective  responses  to  the  use  of  the  vertical  altitude  and  velocity 
indicator 

•  All  testing  will  take  place  in  a  regular  MIT  classroom  setting  or  at  Draper  Laboratory  located  at  One 
Kendal  Square 

•  Total  time  =  less  than  3  hours 

.  POTENTIAL  RISKS  AND  DISCOMFORTS 

There  are  no  anticipated  physical  or  psychological  risks  in  this  study. 

.  POTENTIAL  BENEFITS 

While  there  is  no  immediate  foreseeable  benefit  to  you  as  a  participant  in  this  study,  your  effort  will  provide 
critical  insight  into  the  effectiveness  of  the  vertical  altitude  and  velocity  indicator  (VAVI)  in  providing  an 
ecological  perceptual  aid  for  pilot  performance  during  cognitively  challenging  and  unstable  hover  and  vertical 
descent  operations. 
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PAYMENT  FOR  PARTICIPATION 


Your  participation  is  purely  on  a  volunteer  basis,  thus  no  payment  will  be  given  for  your  participation. 

.  CONFIDENTIALITY 

Any  information  that  is  obtained  in  connection  with  this  study  and  that  can  be  identified  with  you  will  remain 
confidential  and  will  be  disclosed  only  with  your  permission  or  as  required  by  law.  You  will  be  assigned  a 
subject  number  which  will  be  used  on  all  related  documents  to  include  databases,  summaries  of  results,  etc. 
Only  one  master  list  of  subject  names  and  numbers  will  exist  that  will  remain  only  in  the  custody  of  Professor 
Cummings. 

•  IDENTIFICATION  OF  INVESTIGATORS 

If  you  have  any  questions  or  concerns  about  the  research,  please  feel  free  to  contact: 

Missy  Cummings  (Principal  Investigator) 

missvc@mit.edu 

(617)  252-1512 

•  EMERGENCY  CARE  AND  COMPENSATION  FOR  INJURY 

In  the  unlikely  event  of  physical  injury  resulting  from  participation  in  this  research  you  may  receive  medical 
treatment  from  the  M.I.T.  Medical  Department,  including  emergency  treatment  and  follow-up  care  as  needed. 
Your  insurance  carrier  may  be  billed  for  the  cost  of  such  treatment.  M.I.T.  does  not  provide  any  other  form  of 
compensation  for  injury.  Moreover,  in  either  providing  or  making  such  medical  care  available  it  does  not  imply 
the  injury  is  the  fault  of  the  investigator.  Further  information  may  be  obtained  by  calling  the  MIT  Insurance  and 
Legal  Affairs  Office  at  1-617-253  2822. 

.  RIGHTS  OF  RESEARCH  SUBJECTS 

You  are  not  waiving  any  legal  claims,  rights  or  remedies  because  of  your  participation  in  this  research  study.  If 
you  feel  you  have  been  treated  unfairly,  or  you  have  questions  regarding  your  rights  as  a  research  subject,  you 
may  contact  the  Chairman  of  the  Committee  on  the  Use  of  Humans  as  Experimental  Subjects,  M.I.T.,  Room 
E32-335, 77  Massachusetts  Ave,  Cambridge,  MA  02139,  phone  1-617-253  6787. 
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I  SIGNATURE  OF  RESEARCH  SUBJECT  OR  LEGAL  REPRESENTATIVE  1 

I  understand  the  procedures  described  above.  My  questions  have  been  answered  to  my  satisfaction,  and  I  agree 
to  participate  in  this  study.  I  have  been  given  a  copy  of  this  form. 

Name  of  Subject 

Name  of  Legal  Representative  (if  applicable) 

Signature  of  Subject  or  Legal  Representative  Date 


|  SIGNATURE  OF  INVESTIGATOR 


In  my  judgment  the  subject  is  voluntarily  and  knowingly  giving  informed  consent  and  possesses  the  legal 
capacity  to  give  informed  consent  to  participate  in  this  research  study. 


Signature  of  Investigator 


Date 
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Appendix  C 


Demographic  Survey 
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An  Ecological  Perceptual  Aid  for  Lunar  and  Earth- 
Based  Vertical  Precision  Landing 


SUBJECT 

DATE 

TIME 

Demographic  Survey 


1.  Age: 

a  <21 

□  2!  -40 

□  40-60 

□  60  + 

2.  Gender:  aMale  □  Female 

3.  Occupation: _ 

4.  Do  you  have  experience  flying  fixed-wing  or  rotary  aircraft?  o  Yes  □  No 

If  yes: 

a)  Type: 

□  Rotary  Aircraft 

□  Fixed-Wing  Aircraft 

□  Other: _ 

b)  Number  of  hours: _ _ 

5.  Do  you  have  experience  flying  PC-based  aircraft  simulators?  o  Yes  □  No 

If  yes: 

a)  Software  Package: _ 

b)  Total  number  of  simulator  hours: _ 

c)  Which  aircraft  platform  do  you  primarily  fly? 

□  Rotary  Aircraft 

□  Fixed-Wing  Aircraft 

□  Other: _ 

d)  How  often  do  you  fly  the  simulator? 

□  Less  than  1  hour  per  week 

□  Between  1  and  4  hours  per  week 

□  Between  1  and  2  hours  per  day 

□  More  than  2  hours  per  day 

6.  Do  vou  have  experience  flying  a  non  PC-based  aircraft  simulator?  oYes  dNo 

If  yes: 

a)  Aircraft  Platform: _ 

b)  Total  number  of  simulator  hours: _ 

c)  How  often  do  you  fly  the  simulator? 

□  1-10  hours  per  6  months 
a  <  1  hour  per  6  months 

□  >  10  hours  per  6  months 

7.  How  often  do  you  play  other  video  games? 

□  Never 

□  Less  than  1  hour  per  week 

□  Between  1  and  4  hours  per  week 

□  Between  1  and  2  hours  per  day 

□  More  than  2  hours  per  day 

8.  Are  you  color  blind?  □  Yes  a  No 

If  yes: 

a)  Type: 

a  Red/Green 
o  Blue/Yellow 

□  Fully 

□  Other: _ 


124 


Appendix  D 


Experiment  Powerpoint  Tutorial 


D.l  VAVI  HUD  Tutorial 

The  following  experiment  tuturial  was  seen  by  participants  in  the  VAVI  HUD  flight 
display  group. 
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Experiment  Tutorial 


Please  click  to 
continue _ 


Experiment  Explanation 


Thank  you  very  much  for  participating  In  this  experiment. 
Today  you  will  be  tasked  with  flying  an  AV-8B  Harrier  simulator 
using  Microsoft  Flight  Simulator  2004.  You  will  be  asked  to  takeoff, 
fly  the  Harrier,  hover,  and  land  at  various  locations  around  the 
country.  Your  participation  in  this  experiment  will  help  with  the 
testing  of  a  precision  landing  Instrument  to  be  used  on  hover 
aircraft  and/or  spacecraft  such  as  a  lunar  lander. 

You  will  be  using  a  Joystick  and  throttle  control  that  simulates 
the  thrust,  and  pitch  and  roll  of  a  real  harrier.  You  will  have  some 
time  to  train  and  practice  flying  the  harrier  before  moving  on  to 
four  short  test  scenarios.  Jn  order  to  move  on  to  tne  test 
scenarios,  you  must  demonstrate  a  certain  level  of  proficiency 
during  the  training  session.  Do  not  be  concerned  about  not  being 
able  to  reach  this  level  of  proficiency.  It  is  merely  a  way  to  remove 
some  of  the  bias  associated  with  using  subjects  of  all  different 
levels  of  experience.  You  will  have  30  minutes  of  training  available 
to  reach  this  proficiency  level  and  the  majority  of  you  will  have  no 
problem  achieving  this. 


The  following  slides  cover  some  basics 
about  flying  an  AV-8B  Harrier.  Please  take 
the  time  to  read  through  the  information 
thoroughly. 


The  Boeing/BAe  AV-8B  Harrier  II  is  Short 
Takeoff  and  Vertical  Landing  (STOVL) 
single-seat  multi-role  attack  aircraft. 
Several  Marine  squadrons  have  been 
equipped  with  the  AV-8B. 


Thrust  Vectoring  Nozzles 

•  In  a  Harrier,  vectored  thrust  is  used  to  fly  at  low 
speeds 

•  The  jet  exhaust  of  the  engine  is  directed 
downward  via  rotating  nozzles  so  that  thrust 
counters  gravity 

•  There  are  10  nozzle  positions 

-  Position  0:  Nozzles  directed  backward  (for  normal 
flight) 

-  Position  9:  Nozzles  directed  downward  (for  vectored- 
thrust  flight) 

•  ‘Puffers’  take  in  air  that  is  bled  from  the  engine 
and  are  used  to  keep  the  aircraft  level 
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Display  Description 

In  the  following  slides  you  will  see  screenshots  of 
the  Boelng/BAe  AV-8B  Harrier  Head-Up  Display 
(HUD)  that  was  created  for  this  experiment.  The 
HUD  that  you  will  use  to  fly  the  Harrier  includes 
typical  HUD  symbology  with  the  exception  of 
altitude  and  vertical  speed.  Altitude  and  vertical 
speed  are  indicated  using  the  "VAVI"  (Vertical 
Altitude  and  Velocity  Indicator).  The  following 
screen  shots  will  explain  all  of  the  symbology, 
including  the  VAVI. 


Flight  Controls 

Throttle 

-  Controls  thrust  for  normal  flight 

-  Controls  thrust  for  vectored-thrust  flight  at  low  speeds 
Ailerons 

-  Controls  bank  angle 
Elevators 

-  Controls  pitch  angle 
Rudders 

-  Controls  yaw  and  helps  maintain  stability  during 
vectored-thrust  flight  at  low  speeds 


Questions? 

•  Any  questions  about  what  you  have 
seen  thus  far? 

•  You  will  also  have  the  opportunity  to 
ask  questions  in  the  practice  session 
and  the  instructions  presented  here 
will  become  clearer  as  you  practice 
flying 


We  will  now  go  through  a  series  of  lessons 
for  learning  how  to  appropriately  fly  the 
Harrier.  After  each  lesson  you  will  have 
an  opportunity  to  practice  the  task  multiple 
times  until  you  are  proficient  at  the  task. 
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Thank  you. 

You  are  now  ready  to  begin 
practicing... 

GOOD  LUCK! 


Lesson  #1a:  Vertical  Takeoff 

•  Hold  the  brakes 

•  Nozzle  position  9 

•  Increase  throttle  to  full  throttle  and  release 
the  brakes 

•  Your  altitude  will  very  slowly  increase 

•  Keep  pitch  angle  slightly  positive  using  the 
elevator  control 


Lesson  #1  b:  Stationary  Hover 

•  Performed  during  vertical  takeoff  or 
landing 

•  Maintain  a  level  attitude 

•  Use  throttle  control  to  bring  vertical  speed 
to  zero  and  maintain  desired  altitude 


Training  Task  1 

•  Perform  a  vertical  takeoff  from  Green 
State  airport  in  Providence,  Rl 

•  When  you  reach  50  ft  altitude,  maintain  a 
50  ft  hover  for  30  seconds 


Training  Task  1 


Lesson  #2a:  Conventional  Takeoff 

•  Nozzle  position  0 

•  Full  thrust 

•  Aircraft  rotates  at  200  knots  (i.e.  pitch  up 
at  200  knots) 

•  Remember  to  raise  your  landing  gear 
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Lesson  #2b:  Cruising 

Maintain  safe  pitch  and  bank  angles  using  the 

aileron  and  elevator  control 

When  the  aircraft  reference  symbol  is  level  with 

the  horizon,  the  Harrier  is  actually  slightly 

pitched  up  and  therefore  will  climb  (this  is  an 

artifact  of  the  aircraft  configuration  on  the 

ground) 

You  can  trim  the  elevators  if  you'd  like  in  order  to 
maintain  a  more  stable  altitude 


Lesson  #2c:  Vertical  Landing  I 

When  you’re  within  sight  of  the  runway,  decrease  speed 
to  <  150  knots  by  throttling  down  and  using  the  airbrakes 
Lower  the  landing  gear  at  <  200  knots 
Increase  nozzle  position  incrementally  and  slowly, 
gainino  stability  after  each  nozzle  increase,  all  the  way  to 
position  9  to  get  desired  approach  speed  (s  40  knots) 

Be  prepared  for  a  drastic  pitch  up  and  increase  in 
altitude  when  you  first  increase  the  nozzle  position 
Your  target  is  to  be  at  1 50  ft  AGL  and  nozzle  position  8 
or  9  directly  over  the  target  landing  site. 


Lesson  #2c:  Vertical  Landing  II 

•  Avoid  steep  banking  with  ailerons  and  use 
rudders  in  conjunction  with  the  ailerons  to 
maintain  stability  and  steering  control  of  aircraft 

-  tf  aircraft  is  banking  left,  use  right  rudder 

-  If  aircraft  is  banking  right,  use  left  rudder 

•  The  throttle  now  controls  altitude 

•  Use  throttle  and  elevators  to  control  descent  rate 
and  pitch  angle 

•  You  want  to  maintain  a  level  pitch 

•  When  over  landing  spot,  gently  throttle  back  for 
a  soft  touch-down 


Vectored-Thrust  Flight  Tips 

Use  vectored-thrust  only  at  150  knots  or  less 
Transition  into  nozzle  position  9  no  higher  than  150  ft 
The  higher  the  nozzle  position,  the  more  throttle  needed 
to  remain  airborne 

There  is  NO  flaps  control  in  the  Harrier  simulator 
Gentle  control  movements!  Severe  movements  will 
cause  instability! 

Transitions  cause  the  aircraft  to  pitch  up.  Be  ready  and 
make  gentle  corrections 

Don’t  forget  to  raise  and  lower  your  landing  gear! 


Training  Task  2 

Perform  a  conventional  takeoff  from  Green  State 
airport  in  Providence,  Rl 

Cruise  between  1000-2000  ft  AGL  altitude  and  < 
450  knots  at  heading  041  to  the  Boston  Logan 
airport 

Boston  airport  will  be  marked  by  two  red  lights 
on  either  side  of  the  two  main  runways 
Your  target  landing  site  is  a  helipad  located 
between  the  two  main  runways  and  marked  by  a 
white  circle  with  a  large  “H"  in  the  center 
Perform  a  vertical  descent  to  landing 


Training  Task  2 


HDG  =  041 
1000-2000  ft  AGL 
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Great  Job!  You  have 
successfully  passed 
the  training  sessions! 

•  Now  that  you  have  completed  your 
training,  you  will  participate  in  2  short  test 
scenarios 

•  At  the  completion  of  each  test  scenario, 
you  will  be  asked  a  series  of  question  on 
paper  (in  flow-chart  form)  that  will  assess 
your  experience  flying 


Flight  Objectives  (1) 

•  The  following  tasks  are  the  key  goals  of  this 
experiment  and  should  be  performed  to  the  best 
of  your  ability.  They  are  in  no  specific  order 

•  Primary  Goals: 

-  Land  at  the  target  landing  site 

-  Capture  the  commanded  hover  altitude  as  precisely 
as  possible 

-  Maintain  an  accurate  commanded  hover  altitude 

-  Maintain  an  accurate  commanded  sink  rate 

•  Secondary  Goal: 

-  Minimize  horizontal  movement  during  a  vertical 
descent 


Flight  Objectives  (2) 

Test  Scenario  #1 

•  The  following  tasks  are  NOT  of 

Importance  in  this  experiment 
-Takeoff 

-  Straight  and  level  flight 

•  Perform  a  conventional  takeoff  from  Providence 

•  Maintain  a  heading  of  041  at  1000  -  2000  ft  radar 
altitude  and  <  450  knots  to  the  Boston  airport 

•  Slow  to  a  hover  at  50  feet  altitude  above  the 
helipad  or  when  you  stable  out  and  maintain  the 
hover  for  20  seconds.  Please  tell  me  when  you 
are  starting  your  hover  and  do  not  try  to  climb 
back  up  to  the  hover  altitude  if  you  miss  it. 

•  Perform  a  vertical  landing  to  the  helipad  while 
maintaining  a  dynamic  sink  rate  less  than  your 
current  altitude  (i.e.  if  you're  at  150  feet,  don't 
descend  any  faster  than  150  fpm,  etc.) 

Test  Scenario  #2 

•  Perform  a  conventional  takeoff  from  Providence 

•  Maintain  a  heading  of  041  at  1000  -  2000  ft  radar 
altitude  and  <  450  knots  to  the  Boston  airport 

•  Slow  to  a  hover  at  100  feet  altitude  above  the 
helipad  or  when  you  stable  out  and  maintain  the 
hover  for  20  seconds.  Please  tell  me  when  you 
are  starting  your  hover  and  do  not  try  to  climb 
back  up  to  the  hover  altitude  if  you  miss  it. 

•  Perform  a  vertical  landing  to  the  helipad  while 
maintaining  a  constant  sink  rate  =  100  fpm 


1 


132 


D.2  Conventional  HUD  Tutorial 


The  following  experiment  tuturial  was  seen  by  participants  in  the  Conventional  HUD 
flight  display  group. 
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Experiment  Tutorial 


FI  ease  click  to 


Experiment  Explanation 


Thank  you  very  much  for  participating  in  this  experiment. 
Today  you  will  be  tasked  with  flying  an  AV-8B  Harrier  simulator 
using  Microsoft  Flight  Simulator  2004,  You  will  be  asked  to  takeoff, 
fly  tne  Harrier,  hover,  and  land  at  various  locations  around  the 
country.  Your  participation  in  this  experiment  will  help  with  the 
testing  of  a  precision  landing  Instrument  to  be  used  on  hover 
aircraft  and/or  spacecraft  such  as  a  lunar  lander. 

You  will  be  using  a  joystick  and  throttle  control  that  simulates 
the  thrust,  and  pitch  and  roll  of  a  real  harrier.  You  will  have  some 
time  to  train  and  practice  flying  the  harrier  before  moving  on  to 
four  short  test  scenarios.  In  order  to  move  on  to  the  test 
scenarios,  you  must  demonstrate  a  certain  level  of  proficiency 
during  the  training  session.  Do  not  be  concerned  about  not  being 
able  to  reach  this  level  of  proficiency.  It  Is  merely  a  way  to  remove 
some  of  the  bias  associated  with  using  subjects  of  all  different 
levels  of  experience.  You  will  have  30  minutes  of  training  available 
to  reach  this  proficiency  level  and  the  majority  of  you  will  have  no 
problem  achieving  this. 


The  following  slides  cover  some  basics 
about  flying  an  AV-8B  Harrier.  Please  take 
the  time  to  read  through  the  information 
thoroughly. 


The  Boeing/BAe  AV-8B  Harrier  II  is  Short 
Takeoff  and  Vertical  Landing  (STOVL) 
single-seat  multi-role  attack  aircraft. 
Several  Marine  squadrons  have  been 
equipped  with  the  AV-8B. 


Thrust  Vectoring  Nozzles 

•  In  a  Harrier,  vectored  thrust  is  used  to  fly  at  low 
speeds 

•  The  jet  exhaust  of  the  engine  is  directed 
downward  via  rotating  nozzles  so  that  thrust 
counters  gravity 

•  There  are  10  nozzle  positions 

-  Position  0:  Nozzles  directed  backward  (for  normal 
flight) 

-  Position  9:  Nozzles  directed  downward  (for  vectored- 
thrust  flight) 

•  “Puffers"  take  in  air  that  is  bled  from  the  engine 
and  are  used  to  keep  the  aircraft  level 
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Display  Description 


In  the  following  slides  you  will  see  screenshots  of 
the  Boeing/BAe  AV-8B  Harrier  Head-Up  Display 
(HUD)  that  was  created  for  this  experiment.  The 
HUD  that  you  will  use  to  fly  the  Harrier  includes 
typical  HUD  symbology  The  following  screen 
shots  will  explain  all  of  the  relative  symbology. 
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Pitch  Ladder  Description 


Flight  Controls 

•  Throttle 

-  Controls  thrust  for  normal  flight 

-  Controls  thrust  for  vectored- thrust  flight  at  low  speeds 

•  Ailerons 

-  Controls  bank  angle 

•  Elevators 

-  Controls  pitch  angle 

•  Rudders 

-  Controls  yaw  and  maintains  stability  during  vectored- 
thrust  flight  at  low  speeds 


Joystick 
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Rudder  Pedals 

Questions? 

•  Any  questions  about  what  you  have 

seen  thus  far? 

•  You  will  also  have  the  opportunity  to 

Hitate'  note's  V  ,v  \ 

ask  questions  in  the  practice  session 
and  the  instructions  presented  here 

will  become  clearer  as  you  practice 
flying 

We  will  now  go  through  a  series  of  lessons 
for  learning  how  to  appropriately  fly  the 
Harrier.  After  each  lesson  you  will  have 
an  opportunity  to  practice  the  task  multiple 
times  until  you  are  proficient  at  the  task. 


Thank  you. 

You  are  now  ready  to  begin 
practicing... 

GOOD  LUCK! 


Lesson  #1  a:  Vertical  Takeoff 

•  Hold  the  brakes 

•  Nozzle  position  9 

•  Increase  throttle  to  full  throttle  and  release 
the  brakes 

•  Your  altitude  will  very  slowly  increase 

•  Keep  pitch  angle  slightly  positive  using  the 
elevator  control 


Lesson  #1  b:  Stationary  Hover 

•  Performed  during  vertical  takeoff  or 
landing 

•  Maintain  a  level  attitude 

•  Use  throttle  control  to  bring  vertical  speed 
to  zero  and  maintain  desired  altitude 
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Training  Task  1 

•  Perform  a  vertical  takeoff  from  Green 
State  airport  in  Providence,  Rl 

•  When  you  reach  50  ft  altitude,  maintain  a 
50  ft  hover  for  30  seconds 


Lesson  #2a:  Conventional  Takeoff 

•  Nozzle  position  0 

•  Full  thrust 

•  Aircraft  rotates  at  200  knots  (i.e.  pitch  up 
at  200  knots) 

•  Remember  to  raise  your  landing  gear 


Lesson  #2b:  Cruising 

•  Maintain  safe  pitch  and  bank  angles  using  the 
aileron  and  elevator  control 

•  When  the  aircraft  reference  symbol  is  level  with 
the  horizon,  the  Harrier  is  actually  slightly 
pitched  up  and  therefore  will  climb  (this  is  an 
artifact  of  the  aircraft  configuration  on  the 
ground) 

•  You  can  trim  the  elevators  if  you'd  like  in  order  to 
maintain  a  more  stable  altitude 


Lesson  #2c:  Vertical  Landing  I 

When  you're  within  sight  of  the  runway,  decrease  speed 
to  <  150  knots  by  throttling  down  and  using  the  airbrakes 
Lower  the  landing  gear  at  <  200  knots 
Increase  nozzle  position  Incrementally  and  slowly, 
gaining  stability  after  each  nozzle  increase,  all  the  way  to 
position  9  to  get  desired  approach  speed  (£  40  knots) 

Be  prepared  for  a  drastic  pitch  up  and  increase  in 
altitude  when  you  first  increase  the  nozzle  position 
Your  target  is  to  be  at  150  ft  AGL  and  nozzle  position  8 
or  9  directly  over  the  target  landing  site. 


Lesson  #2c:  Vertical  Landing  II 

•  Avoid  steep  banking  with  ailerons  and  use 
rudders  in  conjunction  with  the  ailerons  to 
maintain  stability  and  steering  control  of  aircraft 

-  If  aircraft  is  banking  tett,  use  right  rudder 

-  if  aircraft  is  banking  right,  use  left  rudder 

•  The  throttle  now  controls  altitude 

•  Use  throttle  and  elevators  to  control  descent  rate 
and  pitch  angle 

•  You  want  to  maintain  a  level  pitch 

•  When  over  landing  spot,  gently  throttle  back  for 
a  soft  touch-down 


Vectored-Thrust  Flight  Tips 

Use  vectored-thrust  only  at  1 50  knots  or  less 
Transition  into  nozzle  position  9  no  higher  than  1 50  ft 
The  higher  the  nozzle  position,  the  more  throttle  needed 
to  remain  airborne 

There  is  NO  flaps  control  in  the  Harrier  simulator 
Gentle  control  movements’  Severe  movements  will 
cause  instability! 

Transitions  cause  the  aircraft  to  pitch  up.  Be  ready  and 
make  gentle  corrections 

Don’t  forget  to  raise  and  lower  your  landing  gear! 


Training  Task  2 

•  Perform  a  conventional  takeoff  from  Green  State 
airport  in  Providence,  Rl 

■  Cruise  between  1000-2000  ft  AGL  altitude  and  < 
450  knots  at  heading  041  to  the  Boston  Logan 
airport 

•  Boston  airport  will  be  marked  by  two  red  lights 
on  either  side  of  the  two  main  runways 

■  Your  target  landing  site  is  a  helipad  located 
between  the  two  main  runways  and  marked  by  a 
white  circle  with  a  large  "H"  in  the  center 

•  Perform  a  vertical  descent  to  landing 


®  Great  Job!  You  have 
successfully  passed 
the  training  sessions! 

•  Now  that  you  have  completed  your 
training,  you  will  participate  in  2  short  test 
scenarios 

•  At  the  completion  of  each  test  scenario, 
you  will  be  asked  a  series  of  question  on 
paper  (in  flow-chart  form)  that  will  assess 
your  experience  flying 


Flight  Objectives  (1) 

The  following  tasks  are  the  key  goals  of  this 
experiment  and  should  be  performed  to  the  best 
of  your  ability.  They  are  in  no  specific  order 
Primary  Goals: 

-  Land  at  the  target  landing  site 

-  Capture  the  commanded  hover  altitude  as  precisely 
as  possible 

-  Maintain  an  accurate  commanded  hover  altitude 

-  Maintain  an  accurate  commanded  sink  rate 
Secondary  Goal: 

-  Minimize  horizontal  movement  during  a  vertical 
descent 


Flight  Objectives  (2) 

•  The  following  tasks  are  NOT  of 
Importance  in  this  experiment 
-Takeoff 

-  Straight  and  level  flight 
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Test  Scenario  #1 

Perform  a  conventional  takeoff  from  Providence 
Maintain  a  heading  of  041  at  1000  -  2000  ft  radar 
altitude  and  <  450  knots  to  the  Boston  airport 
Slow  to  a  hover  at  50  feet  altitude  above  the 
helipad  or  when  you  stable  out  and  maintain  the 
hover  for  20  seconds.  Please  tell  me  when  you 
are  starting  your  hover  and  do  not  try  to  climb 
back  up  to  the  hover  altitude  if  you  miss  it. 
Perform  a  vertical  landing  to  the  helipad  while 
maintaining  a  dynamic  sink  rate  less  than  your 
current  altitude  (i.e.  if  you're  at  150  feet,  don't 
descend  any  faster  than  150  fpm,  etc.) 


Test  Scenario  #2 

•  Perform  a  conventional  takeoff  from  Providence 

■  Maintain  a  heading  of  041  at  1000  -  2000  ft  radar 
altitude  and  <  450  knots  to  the  Boston  airport 

•  Slow  to  a  hover  at  100  feet  altitude  above  the 
helipad  or  when  you  stable  out  and  maintain  the 
hover  for  20  seconds.  Please  teli  me  when  you 
are  starting  your  hover  and  do  not  try  to  climb 
back  up  to  the  hover  altitude  if  you  miss  it. 

•  Perform  a  vertical  landing  to  the  helipad  while 
maintaining  a  constant  sink  rate  =  100  fpm 


1 
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Appendix  E 


Semi- Structured  Interview 
Questions 


This  appendix  outlines  the  semi-structured  interview  questions. 

E.l  VAVI  HUD  Questions 

The  following  questions  were  used  to  guide  post-experiment  discussion  with  the  par¬ 
ticipants  who  used  the  VAVI  HUD. 

1.  Did  you  use  the  VAVI?  If  so,  how? 

2.  Was  there  anything  negative  or  distracting  about  the  VAVI? 

3.  Is  there  any  additional  information  that  you  wished  you  had  that  you  think 
would  have  helped  with  the  vertical  descent? 


E.2  Conventional  HUD  Questions 

The  following  questions  were  used  to  guide  post-experiment  discussion  with  the  par¬ 
ticipants  who  used  the  conventional  HUD. 

1.  What  process  did  you  go  through  to  determine  your  vertical  velocity  and  posi¬ 
tion  relative  to  where  you  wanted  to  be? 
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2.  Did  you  find  these  tasks  difficult?  If  so,  what  was  difficult  about  them? 

3.  Is  there  any  additional  information  that  you  wished  you  had  that  you  think 
would  have  helped  with  the  vertical  descent? 
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Appendix  F 


Descriptive  Statistics 


This  appendix  outlines  the  demographics  of  the  particpant  pool  used  in  the  human 
peformance  experimentation. 


F.l  Demographics 


Table  F.l:  Study  Demographics 


Category 

N 

Min 

Max 

Mean 

Std.  Dev. 

Age  (years) 

22 

12 

63 

34 

14 

Rotary  Wing  Experience  (hours) 

11 

15 

9000 

1858 

2908 

Fixed  Wing  Experience  (hours) 

11 

4 

5000 

889 

1492 

Flight  Simulator  Experience  (hours) 

19 

10 

400 

100 

125 

Student  (Y/N) 

6 

- 

- 

- 

- 

Gender  (M/F) 

20M,2F 

- 

- 

- 

- 

F.2  Top  Performer  Non-Parametric  Summary 
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Table  F.2:  Top  Performer  Non-Parametric  Summary 


Dep.  Measure 

Mann- Whitney 

Wilcoxon 

HUD 

U 

z 

VS  Heuristic 

z 

Hover  Accuracy 

p  =  0.589 

14.00 

-0.647 

N/A 

N/A 

Hover  Precision 

p  =  0.126 

6.00 

-1.643 

N/A 

N/A 

VS  Precision 

p  =  0.015 

3.00 

-2.402 

p  =  0.600 

-0.524 

Descent  Duration  Error 

p  =  0.699 

15.00 

-0.480 

p  =  0.028  - 

-2.201 

10-pt  Workload 

p  =  0.026 

4.00 

-2.407 

p  =  0.157  • 

-1.414 
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Appendix  G 


Statistical  Tests 


G.l  Analysis  of  Variance  (ANOVA) 

Analysis  of  variance  tests  are  used  to  test  the  difference  between  the  means  of  two 
or  more  groups.  The  groups  correspond  to  the  different  levels  of  the  independent 
variables.  Each  factor  level  has  a  probability  distribution  of  responses.  The  ANOVA 
model  makes  the  following  assumptions  about  those  probability  distributions  [51]: 

1.  Each  probability  distribution  is  normal. 

2.  Each  probability  distribution  has  the  same  variance. 

3.  The  response  for  each  factor  level  are  random  selections  from  the  corresponding 
probability  distribution  and  are  independent  of  the  responses  for  any  other 
factor  level. 

G.1.1  Single  Factor 

Single  factor  or  one-way  ANOVA  test  the  difference  between  groups  that  are  classified 
by  a  single  independent  variable.  The  basic  model  for  a  one-way  ANOVA  is  outlined 
in  Equation  G.l  where  i  is  the  factor  level  and  j  is  the  number  of  cases. 

Yij  —  /q  -f-  €ij  (G.l) 
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G.1.2  Multiple  Factor 


Multiple  factor  ANOVA  tests  the  effects  of  two  or  more  factors  simultaneously.  Multi¬ 
ple  factor  studies  are  good  because  they  parse  out  any  interaction  between  the  factors 
meaning  they  determine  any  joint  effects  of  the  two  factors.  Multiple  factor  studies 
are  often  more  efficient  and  can  strengthen  the  validity  of  the  findings  [51].  The  basic 
model  for  a  multiple  factor  ANOVA  is  outlined  in  Equation  G.2  where  i  is  the  first 
factor  level,  j  is  2nd  factor  level,  and  k  is  the  observation  from  the  number  of  cases. 

Vjfc  =  l^ij  T  £ijk  (G.2) 

G.2  Kolmogorov-Smirnov  Test  of  Normality 

The  Kolmogorov-Smirnov  (KS)  test  is  used  to  determine  if  a  sample  of  N  data  points 
comes  from  a  population  with  a  specific  distribution.  In  the  context  of  this  study, 
the  KS  test  is  used  to  determine  if  samples  come  from  a  population  with  a  normal 
cumulative  distribution.  The  KS  test  is  a  very  conservative  test  in  terms  of  checking 
for  normality.  To  use  the  test,  a  D  statistic  is  computed  in  the  following  manner. 

D  =  Max  F(Yi)  -  ±  (G.3) 

F  is  the  theoretical  cumulative  distribution  of  the  distribution  being  tested,  which 
must  be  continuous  and  fully  defined  in  terms  of  a  mean  and  standard  deviation.  A 
comparison  of  this  D  statistic  to  a  critical  D  value  from  a  published  table  determines 
the  normality  of  the  distribution.  If  the  D  statistic  is  greater  than  the  critical  value, 
it  is  concluded  that  the  distribution  is  not  normal.  Therefore,  normality  is  shown  by 
a  non-significant  results  in  this  test.  In  addition  to  requiring  that  the  distribution  be 
continuous,  the  KS  test  can  be  more  sensitive  near  the  center  of  the  distribution  than 
at  the  tails.  Despite  these  minor  disadvantages  of  the  test,  it  is  a  very  conservative 
test  that  is  powerful  for  determining  normality  of  a  data  set. 
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G.3  Levene’s  Test  of  Equality  of  Error  Variance 


Levene’s  test  is  used  to  test  if  k  samples  have  equal  variances  in  which  case  there 
would  be  homogeneity  of  variance.  The  Levene  test  tests  the  null  hypothesis  that  all 
variances  are  equal.  Therefore,  to  show  that  the  variances  are  equal,  we  look  for  non¬ 
significance  in  this  test.  The  following  formula  (Equation  G.4)  is  used  to  calculate 
the  Levene  statistic  where  variable  Y  has  a  sample  size  of  N  divided  into  k  subgroups 
and  Ni  is  the  sample  size  of  the  ith  subgroup  [52]. 


(N-k)'£Ni(Zi.-Zi) 

(*-i)E  ZiZij-Zt.y 


G.4  Mann  Whitney  U  Test 

The  Mann- Whitney  U  Test  is  a  non-parametric  test  based  on  rankings  for  deter¬ 
mining  whether  two  samples  come  from  the  same  population  when  the  normality  or 
homogeneity  of  variance  assumptions  are  violated.  In  this  test,  all  the  sample  values 
are  ranked  and  the  sum  of  the  ranks  for  each  sample  is  calculated  and  denoted  by 
Ri  and  R2.  With  samples  sizes  Ni  and  iV2,  the  following  U  statistic  corresponding  to 
sample  1  is  calculated. 


H  =  iVjiVa  + 


N1{N1  +  1) 


-Ri 


(G.5) 


Since  the  sampling  distribution  of  U  is  symmetrical,  the  mean  and  variance  can 
be  defined  as  outlined  below. 


Vu 


N\N2 

2 


(G.6) 


N\N2(N\  +  N2  +  1) 
12 


(G.7) 


From  this  information,  a  z  statistic  can  be  calculated  in  the  following  manner.  If 
both  sample  sizes  are  eight  or  larger,  then  the  U  distribution  is  nearly  normal  with 
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a  mean  of  0  and  a  variance  of  1.  Then,  using  a  z  table,  it  is  possible  to  determine 
whether  the  two  samples  differ  significantly. 


U  —  Hu 
ou 


(G.8) 


G.5  Wilcoxon  Signed-Rank  Test 

The  Wilcoxon  Signed-Rank  Test  is  a  non-parametric  test  which  is  an  alternative  to 
a  paired  t-test  for  comparing  the  differences  between  measurements  when  normal¬ 
ity  assumptions  are  violated.  The  Wilcoxon  test  involves  using  the  magnitudes  and 
signs  of  the  differences  between  paired  observations  to  determine  if  there  is  a  signif¬ 
icant  difference.  The  following  steps  are  taken  to  determine  the  signed-rank  of  the 
observations  [53]. 

1.  take  the  absolute  difference  \Xa  —  for  each  pair; 

2.  omit  from  consideration  those  cases  where  \Xa  —  X(,|=0; 

3.  rank  the  remaining  absolute  differences,  from  smallest  to  largest,  employing  tied 
ranks  where  appropriate; 

4.  assign  to  each  such  rank  a  sign  when  Xa  —  Xb  >  0  and  a  sign  when 
Xa  —  Xb  <  0; 

5.  and  then  calculate  the  value  of  W  for  the  Wilcoxon  test,  which  is  equal  to  the 
sum  of  the  signed  ranks.  The  number  of  signed  ranks,  here  designated  as  n3/r, 
is  equal  to  the  number  of  XaXt,  pairs  at  the  beginning  minus  the  number  of 
pairs  for  which  \Xa  —  W&|=0. 

“When  ns/r  is  equal  to  or  greater  than  10,  the  sampling  distribution  of  W  is  a 
reasonably  close  approximation  of  the  normal  distribution”  [53].  The  corresponding 
p-value  is  found  in  an  appropriate  table. 


150 


Appendix  H 


Top  Performer  Evaluation 


This  appendix  illustrates  the  performance  of  all  of  participants  for  each  of  the  depen¬ 
dent  variables.  The  top  performers  are  circled  and  were  determined  based  on  their 
average  rankings.  Every  participant  from  each  flight  display  group  was  ranked  and 
the  average  of  those  rankings  across  all  dependent  variables  was  used  to  determine 
the  top  performers.  A  low  average  ranking  indicates  consistent  good  performance. 
Table  H.l  outlines  the  average  rankings  for  all  participants  and  highlights  the  top 
performers  in  each  group. 


Table  H.l:  Top  Participants  Average  Ranking 


VAVI 

No  VAVI 

Subject  # 

Avg  Ranking 

Subject  # 

Avg  Ranking 

3 

4.13 

4 

7.00 

6 

4.88 

5 

4.50 

10 

3.38 

11 

125 

15 

4.75 

12 

4.38 

20 

3.00 

14 

3.38 

21 

5.67 

16 

4.75 

23 

6.33 

17 

2.86 

24 

6.12 

29 

6.60 

25 

4.57 

30 

5.13 

27 

3.50 

31 

4.43 

28 

7.50 

- 

- 
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Mean  RMSE  of  hover  Mean  hover  altitude  error  (ft) 
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Mean  descent  duration  error  normalized  Mean  RMSE  of  vertical  speed  during  the  descent 


H.1.2  Landing  Performance 


Figure  H-2:  Top  Participants  Landing  Performance,  VAVI  Flight  Display  Group 
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Mean  RMSE  of  hover  Mean  hover  altitude  error  (ft) 


Conventional  /  Static 


Figure  H-3:  Top  Participants  Hover  Performance,  Conventional  Flight  Display  Group 


1 


Mean  descent  duration  error  normalized  Mean  RMSE  of  vertical  speed  during  the  descent 


;so.oo 


Conventional  /  Static 


300  00 


Conventional  I  Dynamic 


Figure  H-4:  Top  Participants  Landing  Performance,  Conventional  Flight  Display 
Group 
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